BUG #14941: Vacuum crashes
The following bug has been logged on the website:
Bug reference: 14941
Logged by: Lyes Ameddah
Email address: lyes.amd@gmail.com
PostgreSQL version: 9.6.0
Operating system: CentOs 7
Description:
Hello,
I make a complete empty once a week in an automated way and it happens that
the vacuum is stuck on a table (perhaps another process has a lock first).
The behavior I would like to see is that the void ignores this table and
moves to another instead of being blocked.
Is it possible to have a solution to this problem please?
Thank you!
On 12/01/2017 05:09 PM, lyes.amd@gmail.com wrote:
The following bug has been logged on the website:
Bug reference: 14941
Logged by: Lyes Ameddah
Email address: lyes.amd@gmail.com
PostgreSQL version: 9.6.0
The current minor version in 9.6 branch is 9.6. You're missing a year
worth of bugfixes ...
Operating system: CentOs 7
Description:Hello,
I make a complete empty once a week in an automated way and it happens that
the vacuum is stuck on a table (perhaps another process has a lock first).
1) Completely empty what?
2) Do you mean autovacuum or manual vacuum?
3) Do you see waiting locks in pg_locks catalog while this is happening?
SELECT * FROM pg_locks WHERE NOT granted;
The behavior I would like to see is that the void ignores this table and
moves to another instead of being blocked.
I believe autovacuum should not block waiting for a heavy-weight lock on
a table since this commit that went into 9.1:
So I'm wondering what problem you're running into.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 12/1/17, 11:16 AM, "Tomas Vondra" <tomas.vondra@2ndquadrant.com> wrote:
The behavior I would like to see is that the void ignores this table and
moves to another instead of being blocked.I believe autovacuum should not block waiting for a heavy-weight lock on
a table since this commit that went into 9.1:So I'm wondering what problem you're running into.
It sounds like Lyes is doing a periodic database-wide VACUUM that is
getting blocked on certain relations that are already locked (perhaps
because autovacuum is already working on it). IIUC, the ask is to provide
a way to skip vacuuming relations that cannot be immediately locked.
There is already an internal flag called VACOPT_NOWAIT that gives
autovacuum this ability in certain cases (introduced in the aforementioned
commit), and I recently brought up this potential use-case as
justification for a patch [0]/messages/by-id/875815E8-7A99-4488-AA07-F254C404E2CF@amazon.com. I'd be happy to submit a patch for
providing VACOPT_NOWAIT to users if others think it's something we should
do.
As a side note, this seems more like a feature request than a bug report,
so I'm adding pgsql-hackers@ here, too.
Nathan
[0]: /messages/by-id/875815E8-7A99-4488-AA07-F254C404E2CF@amazon.com
On Fri, Dec 1, 2017 at 1:53 PM, Bossart, Nathan <bossartn@amazon.com> wrote:
There is already an internal flag called VACOPT_NOWAIT that gives
autovacuum this ability in certain cases (introduced in the aforementioned
commit), and I recently brought up this potential use-case as
justification for a patch [0]. I'd be happy to submit a patch for
providing VACOPT_NOWAIT to users if others think it's something we should
do.
Seems entirely reasonable to me, provided that we only update the
extensible-options syntax:
VACUUM [ ( option [, ...] ) ] [ table_and_columns [, ...] ]
I don't want to add any more options to the older syntax:
VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ table_and_columns [, ...] ]
I am slightly confused as to how we got on to this topic since the
subject is "Vacuum crashes", but perhaps Lyes was simply speaking
imprecisely.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 12/1/17, 3:04 PM, "Robert Haas" <robertmhaas@gmail.com> wrote:
On Fri, Dec 1, 2017 at 1:53 PM, Bossart, Nathan <bossartn@amazon.com> wrote:
There is already an internal flag called VACOPT_NOWAIT that gives
autovacuum this ability in certain cases (introduced in the aforementioned
commit), and I recently brought up this potential use-case as
justification for a patch [0]. I'd be happy to submit a patch for
providing VACOPT_NOWAIT to users if others think it's something we should
do.Seems entirely reasonable to me, provided that we only update the
extensible-options syntax:VACUUM [ ( option [, ...] ) ] [ table_and_columns [, ...] ]
I don't want to add any more options to the older syntax:
VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ table_and_columns [, ...] ]
Right. This seems like the right path forward to me, as the
VACUUM documentation states that the unparenthesized syntax is
deprecated, and the DISABLE_PAGE_SKIPPING option was not added
to the old syntax, either.
I am slightly confused as to how we got on to this topic since the
subject is "Vacuum crashes", but perhaps Lyes was simply speaking
imprecisely.
I'm hoping Lyes chimes in soon to clarify if I am interpreting
the original report incorrectly.
Nathan
sorry guys, yes I'm talking about a FULL VACUUM and not about Auto-Vacuum.
Thank you very match for your feedback.
That's Waht I do :
vacuum FULL VERBOSE ANALYZE;
reindex database postgres;
I would be happy if there is a patch to fix that !
2017-12-01 22:16 GMT+01:00 Bossart, Nathan <bossartn@amazon.com>:
On 12/1/17, 3:04 PM, "Robert Haas" <robertmhaas@gmail.com> wrote:
On Fri, Dec 1, 2017 at 1:53 PM, Bossart, Nathan <bossartn@amazon.com>
wrote:
There is already an internal flag called VACOPT_NOWAIT that gives
autovacuum this ability in certain cases (introduced in theaforementioned
commit), and I recently brought up this potential use-case as
justification for a patch [0]. I'd be happy to submit a patch for
providing VACOPT_NOWAIT to users if others think it's something weshould
do.
Seems entirely reasonable to me, provided that we only update the
extensible-options syntax:VACUUM [ ( option [, ...] ) ] [ table_and_columns [, ...] ]
I don't want to add any more options to the older syntax:
VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ table_and_columns
[, ...] ]
Right. This seems like the right path forward to me, as the
VACUUM documentation states that the unparenthesized syntax is
deprecated, and the DISABLE_PAGE_SKIPPING option was not added
to the old syntax, either.I am slightly confused as to how we got on to this topic since the
subject is "Vacuum crashes", but perhaps Lyes was simply speaking
imprecisely.I'm hoping Lyes chimes in soon to clarify if I am interpreting
the original report incorrectly.Nathan
--
*Lyes AMEDDAH*
*Téléphone portable : 06 66 24 50 70*
*Titre RNCP I - Développement Web*
*HiTema*
On Mon, Dec 4, 2017 at 8:59 AM, Lyes Ameddah <lyes.amd@gmail.com> wrote:
sorry guys, yes I'm talking about a FULL VACUUM and not about Auto-Vacuum.
Thank you very match for your feedback.
OK, but that's not the confusion. What you said is that it CRASHES,
but the behavior you described is that it BLOCKS waiting for a lock.
Blocking and crashing are not the same thing.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 12/4/17, 8:52 AM, "Robert Haas" <robertmhaas@gmail.com> wrote:
On Mon, Dec 4, 2017 at 8:59 AM, Lyes Ameddah <lyes.amd@gmail.com> wrote:
sorry guys, yes I'm talking about a FULL VACUUM and not about Auto-Vacuum.
Thank you very match for your feedback.OK, but that's not the confusion. What you said is that it CRASHES,
but the behavior you described is that it BLOCKS waiting for a lock.
Blocking and crashing are not the same thing.
Provided that Lyes seems to have described wanting to avoid the blocking
behavior (and since I'm interested in adding this functionality anyway),
here are some patches that open up the VACOPT_NOWAIT option to users for
both VACUUM and ANALYZE commands.
---
0001_fix_unparenthesized_vacuum_grammar_v1.patch
One VacuumStmt grammar rule allows users to specify the VACOPT_ANALYZE
option by including an AnalyzeStmt at the end of the command. This
appears to have been added as part of the introduction of the ANALYZE
command (f905d65e). However, this means that users can call VACUUM
commands like
VACUUM VERBOSE ANALYZE VERBOSE pg_class;
Besides being disallowed according to the documentation, I think this
may give users the idea that there is a difference between the VERBOSE
options for VACUUM versus ANALYZE. In fact, they are the same option,
and specifying it twice is redundant.
This change fixes the unparenthesized VACUUM syntax to disallow the out-
of-order VACUUM options as described above. This is a prerequisite
change for opening up VACOPT_NOWAIT to users, as adding it to the
grammar as-is would cause statements like
VACUUM VERBOSE NOWAIT ANALYZE VERBOSE NOWAIT pg_class;
to be allowed. Since I am also looking to introduce a parenthesized
syntax for ANALYZE, this patch would prevent statements like
VACUUM VERBOSE ANALYZE (VERBOSE, NOWAIT) pg_class;
from being accepted as well.
---
0002_add_parenthesized_analyze_syntax_v1.patch
As a prerequisite to giving users access to the VACOPT_NOWAIT option, I
am introducing a parenthesized syntax for ANALYZE that is similar to
VACUUM's. Following VACUUM's example, any new options will be added to
this syntax, and the old style will become deprecated.
Adding a parenthesized syntax for ANALYZE is not strictly necessary for
providing the NOWAIT option, as NOWAIT is already a keyword in gram.y.
However, I thought it would be good to match VACUUM (since the command
structure is so similar), and this work would be required at some point
anyway if ANALYZE ever accepts options that we do not want to make
reserved keywords.
---
0003_add_nowait_vacuum_option_v1.patch
This change provides the existing VACOPT_NOWAIT option to users. This
option was previously only used by autovacuum in special cases, but it
seems like a potentially valuable option in other cases as well. For
example, perhaps a user wants to run a nightly VACUUM job that skips
tables that cannot be locked due to autovacuum (or something else)
already working on it.
I chose to use NOWAIT as the option name because it is already a keyword
for the LOCK command.
This patch follows the existing logging precedent added by 11d8d72c and
ab6eaee8: if a relation is explicitly specified in the command, a log
statement will be emitted if it is skipped. If the relation is not
specified (e.g. "VACUUM FULL;"), no such log statements will be emitted.
---
I'll be adding an entry to the next commitfest for these patches soon.
Nathan
Attachments:
0003_add_nowait_vacuum_option_v1.patchapplication/octet-stream; name=0003_add_nowait_vacuum_option_v1.patchDownload
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 10b3a9a..05882c3 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -27,6 +27,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
VERBOSE
+ NOWAIT
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -77,6 +78,17 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
</varlistentry>
<varlistentry>
+ <term><literal>NOWAIT</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>ANALYZE</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index b760e8e..1d7ffef 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -31,6 +31,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
VERBOSE
ANALYZE
DISABLE_PAGE_SKIPPING
+ NOWAIT
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -161,6 +162,17 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</varlistentry>
<varlistentry>
+ <term><literal>NOWAIT</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
<para>
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f749722..c6cc833 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -10507,6 +10507,7 @@ vacuum_option_elem:
errmsg("unrecognized VACUUM option \"%s\"", $1),
parser_errposition(@1)));
}
+ | NOWAIT { $$ = VACOPT_NOWAIT; }
;
AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
@@ -10534,6 +10535,7 @@ analyze_option_list:
analyze_option_elem:
VERBOSE { $$ = VACOPT_VERBOSE; }
+ | NOWAIT { $$ = VACOPT_NOWAIT; }
;
analyze_keyword:
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2eaa6b2..27e06fb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3116,7 +3116,7 @@ typedef enum VacuumOption
VACOPT_VERBOSE = 1 << 2, /* print progress info */
VACOPT_FREEZE = 1 << 3, /* FREEZE option */
VACOPT_FULL = 1 << 4, /* FULL (non-concurrent) vacuum */
- VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock (autovacuum only) */
+ VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock */
VACOPT_SKIPTOAST = 1 << 6, /* don't process the TOAST table, if any */
VACOPT_DISABLE_PAGE_SKIPPING = 1 << 7 /* don't skip any pages */
} VacuumOption;
diff --git a/src/test/isolation/expected/vacuum-nowait.out b/src/test/isolation/expected/vacuum-nowait.out
new file mode 100644
index 0000000..89c1cf6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-nowait.out
@@ -0,0 +1,64 @@
+Parsed test spec with 2 sessions
+
+starting permutation: lock vac_specified commit
+step lock:
+ BEGIN;
+ LOCK test1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "test1" --- lock not available
+step vac_specified: VACUUM (NOWAIT) test1, test2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_all commit
+step lock:
+ BEGIN;
+ LOCK test1 IN SHARE MODE;
+
+step vac_all: VACUUM (NOWAIT);
+step commit:
+ COMMIT;
+
+
+starting permutation: lock analyze_specified commit
+step lock:
+ BEGIN;
+ LOCK test1 IN SHARE MODE;
+
+WARNING: skipping analyze of "test1" --- lock not available
+step analyze_specified: ANALYZE (NOWAIT) test1, test2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock analyze_all commit
+step lock:
+ BEGIN;
+ LOCK test1 IN SHARE MODE;
+
+step analyze_all: ANALYZE (NOWAIT);
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_analyze_specified commit
+step lock:
+ BEGIN;
+ LOCK test1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "test1" --- lock not available
+step vac_analyze_specified: VACUUM (ANALYZE, NOWAIT) test1, test2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_analyze_all commit
+step lock:
+ BEGIN;
+ LOCK test1 IN SHARE MODE;
+
+step vac_analyze_all: VACUUM (ANALYZE, NOWAIT);
+step commit:
+ COMMIT;
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index e41b916..0ebbdbf 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -63,3 +63,4 @@ test: async-notify
test: vacuum-reltuples
test: timeouts
test: vacuum-concurrent-drop
+test: vacuum-nowait
diff --git a/src/test/isolation/specs/vacuum-nowait.spec b/src/test/isolation/specs/vacuum-nowait.spec
new file mode 100644
index 0000000..cd3d623
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-nowait.spec
@@ -0,0 +1,43 @@
+# Test for the NOWAIT option for VACUUM and ANALYZE commands.
+#
+# If a skipped relation is specified in the command, it should be logged. If a
+# skipped relation was not explicitly specified in the command, it should not be
+# logged.
+
+setup
+{
+ CREATE TABLE test1 (a INT);
+ CREATE TABLE test2 (a INT);
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS test1;
+ DROP TABLE IF EXISTS test2;
+}
+
+session "s1"
+step "lock"
+{
+ BEGIN;
+ LOCK test1 IN SHARE MODE;
+}
+step "commit"
+{
+ COMMIT;
+}
+
+session "s2"
+step "vac_specified" { VACUUM (NOWAIT) test1, test2; }
+step "vac_all" { VACUUM (NOWAIT); }
+step "analyze_specified" { ANALYZE (NOWAIT) test1, test2; }
+step "analyze_all" { ANALYZE (NOWAIT); }
+step "vac_analyze_specified" { VACUUM (ANALYZE, NOWAIT) test1, test2; }
+step "vac_analyze_all" { VACUUM (ANALYZE, NOWAIT); }
+
+permutation "lock" "vac_specified" "commit"
+permutation "lock" "vac_all" "commit"
+permutation "lock" "analyze_specified" "commit"
+permutation "lock" "analyze_all" "commit"
+permutation "lock" "vac_analyze_specified" "commit"
+permutation "lock" "vac_analyze_all" "commit"
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index 199dda6..99aad21 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -113,8 +113,11 @@ ERROR: relation "does_not_exist" does not exist
ANALYZE vactst (i), vacparted (does_not_exist);
ERROR: column "does_not_exist" of relation "vacparted" does not exist
-- parenthesized syntax for ANALYZE
-ANALYZE (VERBOSE) does_not_exist;
+ANALYZE (NOWAIT, VERBOSE) does_not_exist;
ERROR: relation "does_not_exist" does not exist
+-- nowait option
+VACUUM (NOWAIT) vactst;
+ANALYZE (NOWAIT) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index e7b7a7f..5e88c68 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -90,7 +90,11 @@ ANALYZE vactst, does_not_exist, vacparted;
ANALYZE vactst (i), vacparted (does_not_exist);
-- parenthesized syntax for ANALYZE
-ANALYZE (VERBOSE) does_not_exist;
+ANALYZE (NOWAIT, VERBOSE) does_not_exist;
+
+-- nowait option
+VACUUM (NOWAIT) vactst;
+ANALYZE (NOWAIT) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
0002_add_parenthesized_analyze_syntax_v1.patchapplication/octet-stream; name=0002_add_parenthesized_analyze_syntax_v1.patchDownload
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 83b07a0..10b3a9a 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -21,9 +21,14 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
+ANALYZE [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
-<phrase>where <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
+<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
+
+ VERBOSE
+
+<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
<replaceable class="parameter">table_name</replaceable> [ ( <replaceable class="parameter">column_name</replaceable> [, ...] ) ]
</synopsis>
@@ -49,6 +54,13 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
It is further possible to give a list of column names for a table,
in which case only the statistics for those columns are collected.
</para>
+
+ <para>
+ When the option list is surrounded by parentheses, the options can be
+ written in any order. The parenthesized syntax was added in
+ <productname>PostgreSQL</productname> 11; the unparenthesized syntax
+ is deprecated.
+ </para>
</refsect1>
<refsect1>
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 07cc15d..f749722 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -306,6 +306,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> opt_lock lock_type cast_context
%type <ival> vacuum_option_list vacuum_option_elem
+ analyze_option_list analyze_option_elem
%type <boolean> opt_or_replace
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
@@ -10517,6 +10518,22 @@ AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
n->rels = $3;
$$ = (Node *)n;
}
+ | analyze_keyword '(' analyze_option_list ')' opt_vacuum_relation_list
+ {
+ VacuumStmt *n = makeNode(VacuumStmt);
+ n->options = VACOPT_ANALYZE | $3;
+ n->rels = $5;
+ $$ = (Node *) n;
+ }
+ ;
+
+analyze_option_list:
+ analyze_option_elem { $$ = $1; }
+ | analyze_option_list ',' analyze_option_elem { $$ = $1 | $3; }
+ ;
+
+analyze_option_elem:
+ VERBOSE { $$ = VACOPT_VERBOSE; }
;
analyze_keyword:
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index c440c7e..199dda6 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -112,6 +112,9 @@ ANALYZE vactst, does_not_exist, vacparted;
ERROR: relation "does_not_exist" does not exist
ANALYZE vactst (i), vacparted (does_not_exist);
ERROR: column "does_not_exist" of relation "vacparted" does not exist
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 92eaca2..e7b7a7f 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -89,6 +89,9 @@ ANALYZE vacparted (b), vactst;
ANALYZE vactst, does_not_exist, vacparted;
ANALYZE vactst (i), vacparted (does_not_exist);
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
0001_fix_unparenthesized_vacuum_grammar_v1.patchapplication/octet-stream; name=0001_fix_unparenthesized_vacuum_grammar_v1.patchDownload
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index ebfc94f..07cc15d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -437,7 +437,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <boolean> opt_instead
%type <boolean> opt_unique opt_concurrently opt_verbose opt_full
-%type <boolean> opt_freeze opt_default opt_recheck
+%type <boolean> opt_freeze opt_analyze opt_default opt_recheck
%type <defelt> opt_binary opt_oids copy_delimiter
%type <boolean> copy_from opt_program
@@ -10462,7 +10462,7 @@ cluster_index_specification:
*
*****************************************************************************/
-VacuumStmt: VACUUM opt_full opt_freeze opt_verbose opt_vacuum_relation_list
+VacuumStmt: VACUUM opt_full opt_freeze opt_verbose opt_analyze opt_vacuum_relation_list
{
VacuumStmt *n = makeNode(VacuumStmt);
n->options = VACOPT_VACUUM;
@@ -10472,19 +10472,9 @@ VacuumStmt: VACUUM opt_full opt_freeze opt_verbose opt_vacuum_relation_list
n->options |= VACOPT_FREEZE;
if ($4)
n->options |= VACOPT_VERBOSE;
- n->rels = $5;
- $$ = (Node *)n;
- }
- | VACUUM opt_full opt_freeze opt_verbose AnalyzeStmt
- {
- VacuumStmt *n = (VacuumStmt *) $5;
- n->options |= VACOPT_VACUUM;
- if ($2)
- n->options |= VACOPT_FULL;
- if ($3)
- n->options |= VACOPT_FREEZE;
- if ($4)
- n->options |= VACOPT_VERBOSE;
+ if ($5)
+ n->options |= VACOPT_ANALYZE;
+ n->rels = $6;
$$ = (Node *)n;
}
| VACUUM '(' vacuum_option_list ')' opt_vacuum_relation_list
@@ -10534,6 +10524,11 @@ analyze_keyword:
| ANALYSE /* British */ {}
;
+opt_analyze:
+ analyze_keyword { $$ = true; }
+ | /*EMPTY*/ { $$ = false; }
+ ;
+
opt_verbose:
VERBOSE { $$ = true; }
| /*EMPTY*/ { $$ = false; }
On Sat, Dec 2, 2017 at 6:16 AM, Bossart, Nathan <bossartn@amazon.com> wrote:
On 12/1/17, 3:04 PM, "Robert Haas" <robertmhaas@gmail.com> wrote:
Seems entirely reasonable to me, provided that we only update the
extensible-options syntax:VACUUM [ ( option [, ...] ) ] [ table_and_columns [, ...] ]
I don't want to add any more options to the older syntax:
VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ table_and_columns [, ...] ]
Right. This seems like the right path forward to me, as the
VACUUM documentation states that the unparenthesized syntax is
deprecated, and the DISABLE_PAGE_SKIPPING option was not added
to the old syntax, either.
Yeah, the exact same discussion has happened when what has become
DISABLE_PAGE_SKIPPING was discussed for the freeze map.
--
Michael
On Wed, Dec 6, 2017 at 1:52 AM, Bossart, Nathan <bossartn@amazon.com> wrote:
0001_fix_unparenthesized_vacuum_grammar_v1.patch
One VacuumStmt grammar rule allows users to specify the VACOPT_ANALYZE
option by including an AnalyzeStmt at the end of the command. This
appears to have been added as part of the introduction of the ANALYZE
command (f905d65e). However, this means that users can call VACUUM
commands likeVACUUM VERBOSE ANALYZE VERBOSE pg_class;
Besides being disallowed according to the documentation, I think this
may give users the idea that there is a difference between the VERBOSE
options for VACUUM versus ANALYZE. In fact, they are the same option,
and specifying it twice is redundant.
Hmm. Yeah. If adding a parenthesis grammar to ANALYZE this is a good
move for long purposes. If we don't do that now, or at least for the
purpose of this patch set, then a AnalyzeStmt node embedded directly
in the grammar of VACUUM is likely to bite in the future.
This change fixes the unparenthesized VACUUM syntax to disallow the out-
of-order VACUUM options as described above. This is a prerequisite
change for opening up VACOPT_NOWAIT to users, as adding it to the
grammar as-is would cause statements likeVACUUM VERBOSE NOWAIT ANALYZE VERBOSE NOWAIT pg_class;
to be allowed. Since I am also looking to introduce a parenthesized
syntax for ANALYZE, this patch would prevent statements like
If you add only a parenthesized grammar of ANALYZE, it seems to me
that this would not be an allowed query anyway.
VACUUM VERBOSE ANALYZE (VERBOSE, NOWAIT) pg_class;
from being accepted as well.
This could also be equivalent to ANALYZE(VERBOSE, NOWAIT). I can see
pros and cons by keeping the complete extension of AnalyzeStmt in the
VACUUM grammar, but as the documentation does not mention VACUUM
VERBOSE ANALYZE VERBOSE as a legit query I guess that ripping it out
is not going to annoy many users. Still let's see opinions from other
folks on -hackers as another approach to the feature you want to add
here would be to just implement the grammar for VACUUM but *not*
ANALYZE, but I'd like to think that we still want ANALYZE to be
supported, and also we want it to be extensible as a different thing
than what VACUUM is.
0002_add_parenthesized_analyze_syntax_v1.patch
As a prerequisite to giving users access to the VACOPT_NOWAIT option, I
am introducing a parenthesized syntax for ANALYZE that is similar to
VACUUM's. Following VACUUM's example, any new options will be added to
this syntax, and the old style will become deprecated.Adding a parenthesized syntax for ANALYZE is not strictly necessary for
providing the NOWAIT option, as NOWAIT is already a keyword in gram.y.
However, I thought it would be good to match VACUUM (since the command
structure is so similar), and this work would be required at some point
anyway if ANALYZE ever accepts options that we do not want to make
reserved keywords.
Yep, agreed with the contents of this patch.
0003_add_nowait_vacuum_option_v1.patch
This change provides the existing VACOPT_NOWAIT option to users. This
option was previously only used by autovacuum in special cases, but it
seems like a potentially valuable option in other cases as well. For
example, perhaps a user wants to run a nightly VACUUM job that skips
tables that cannot be locked due to autovacuum (or something else)
already working on it.I chose to use NOWAIT as the option name because it is already a keyword
for the LOCK command.
Makes sense to me.
This patch follows the existing logging precedent added by 11d8d72c and
ab6eaee8: if a relation is explicitly specified in the command, a log
statement will be emitted if it is skipped. If the relation is not
specified (e.g. "VACUUM FULL;"), no such log statements will be emitted.
+WARNING: skipping analyze of "test1" --- lock not available
+step analyze_specified: ANALYZE (NOWAIT) test1, test2;
+step commit:
+ COMMIT;
OK for a WARNING for me in this case. We already do that for the other entries.
Let's see what other folks think first about the ANALYZE grammar in
VACUUM, as well as having ANALYZE use a parenthesized grammer. FWIW, I
like the contents of this patch set, and the thing is non-intrusive. I
think that NOWAIT gains more value now that multiple relations can be
vacuumed or analyzed with a single manual command.
--
Michael
On Mon, Dec 11, 2017 at 2:13 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Wed, Dec 6, 2017 at 1:52 AM, Bossart, Nathan <bossartn@amazon.com> wrote:
0001_fix_unparenthesized_vacuum_grammar_v1.patch
One VacuumStmt grammar rule allows users to specify the VACOPT_ANALYZE
option by including an AnalyzeStmt at the end of the command. This
appears to have been added as part of the introduction of the ANALYZE
command (f905d65e). However, this means that users can call VACUUM
commands likeVACUUM VERBOSE ANALYZE VERBOSE pg_class;
Besides being disallowed according to the documentation, I think this
may give users the idea that there is a difference between the VERBOSE
options for VACUUM versus ANALYZE. In fact, they are the same option,
and specifying it twice is redundant.Hmm. Yeah. If adding a parenthesis grammar to ANALYZE this is a good
move for long purposes. If we don't do that now, or at least for the
purpose of this patch set, then a AnalyzeStmt node embedded directly
in the grammar of VACUUM is likely to bite in the future.This change fixes the unparenthesized VACUUM syntax to disallow the out-
of-order VACUUM options as described above. This is a prerequisite
change for opening up VACOPT_NOWAIT to users, as adding it to the
grammar as-is would cause statements likeVACUUM VERBOSE NOWAIT ANALYZE VERBOSE NOWAIT pg_class;
to be allowed. Since I am also looking to introduce a parenthesized
syntax for ANALYZE, this patch would prevent statements likeIf you add only a parenthesized grammar of ANALYZE, it seems to me
that this would not be an allowed query anyway.VACUUM VERBOSE ANALYZE (VERBOSE, NOWAIT) pg_class;
from being accepted as well.
This could also be equivalent to ANALYZE(VERBOSE, NOWAIT). I can see
pros and cons by keeping the complete extension of AnalyzeStmt in the
VACUUM grammar, but as the documentation does not mention VACUUM
VERBOSE ANALYZE VERBOSE as a legit query I guess that ripping it out
is not going to annoy many users. Still let's see opinions from other
folks on -hackers as another approach to the feature you want to add
here would be to just implement the grammar for VACUUM but *not*
ANALYZE, but I'd like to think that we still want ANALYZE to be
supported, and also we want it to be extensible as a different thing
than what VACUUM is.0002_add_parenthesized_analyze_syntax_v1.patch
As a prerequisite to giving users access to the VACOPT_NOWAIT option, I
am introducing a parenthesized syntax for ANALYZE that is similar to
VACUUM's. Following VACUUM's example, any new options will be added to
this syntax, and the old style will become deprecated.Adding a parenthesized syntax for ANALYZE is not strictly necessary for
providing the NOWAIT option, as NOWAIT is already a keyword in gram.y.
However, I thought it would be good to match VACUUM (since the command
structure is so similar), and this work would be required at some point
anyway if ANALYZE ever accepts options that we do not want to make
reserved keywords.Yep, agreed with the contents of this patch.
0003_add_nowait_vacuum_option_v1.patch
This change provides the existing VACOPT_NOWAIT option to users. This
option was previously only used by autovacuum in special cases, but it
seems like a potentially valuable option in other cases as well. For
example, perhaps a user wants to run a nightly VACUUM job that skips
tables that cannot be locked due to autovacuum (or something else)
already working on it.I chose to use NOWAIT as the option name because it is already a keyword
for the LOCK command.Makes sense to me.
This patch follows the existing logging precedent added by 11d8d72c and
ab6eaee8: if a relation is explicitly specified in the command, a log
statement will be emitted if it is skipped. If the relation is not
specified (e.g. "VACUUM FULL;"), no such log statements will be emitted.+WARNING: skipping analyze of "test1" --- lock not available +step analyze_specified: ANALYZE (NOWAIT) test1, test2; +step commit: + COMMIT; OK for a WARNING for me in this case. We already do that for the other entries.
I also reviewed the patches.
For 0003 patch, if we do VACUUM (NOWAIT) to whole database we don't
get the above WARNING messages, but I think we should get them. The
reported issue also did VACUUM FULL and REINDEX to whole database.
Especially when VACUUM to whole database the information of skipped
relations would be useful for users.
Regards,
--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On 12/10/17, 11:13 PM, "Michael Paquier" <michael.paquier@gmail.com> wrote:
Let's see what other folks think first about the ANALYZE grammar in
VACUUM, as well as having ANALYZE use a parenthesized grammer. FWIW, I
like the contents of this patch set, and the thing is non-intrusive. I
think that NOWAIT gains more value now that multiple relations can be
vacuumed or analyzed with a single manual command.
Thanks for taking a look.
Nathan
On 12/11/17, 9:41 PM, "Masahiko Sawada" <sawada.mshk@gmail.com> wrote:
I also reviewed the patches.
Thanks for taking a look.
For 0003 patch, if we do VACUUM (NOWAIT) to whole database we don't
get the above WARNING messages, but I think we should get them. The
reported issue also did VACUUM FULL and REINDEX to whole database.
Especially when VACUUM to whole database the information of skipped
relations would be useful for users.
Perhaps we could attempt to construct the RangeVar just before
logging in this case.
Nathan
On 12/11/17, 9:41 PM, "Masahiko Sawada" <sawada.mshk@gmail.com> wrote:
For 0003 patch, if we do VACUUM (NOWAIT) to whole database we don't
get the above WARNING messages, but I think we should get them. The
reported issue also did VACUUM FULL and REINDEX to whole database.
Especially when VACUUM to whole database the information of skipped
relations would be useful for users.
Here is a new set of patches. 0001 and 0002 are essentially the same
as before except for a rebase and some small indentation fixes. 0003
now includes logic to log all skipped relations regardless of whether
they were specified in the command. I've also modified the isolation
test slightly to use partitioned tables instead of running VACUUM and
ANALYZE on the whole database. This helps prevent timeouts on build-
farm machines with CLOBBER_CACHE_ALWAYS enabled (like with 0a3edbb3).
Nathan
Attachments:
0001_fix_unparenthesized_vacuum_grammar_v2.patchapplication/octet-stream; name=0001_fix_unparenthesized_vacuum_grammar_v2.patchDownload
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index ebfc94f..3c76bda 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -437,7 +437,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <boolean> opt_instead
%type <boolean> opt_unique opt_concurrently opt_verbose opt_full
-%type <boolean> opt_freeze opt_default opt_recheck
+%type <boolean> opt_freeze opt_analyze opt_default opt_recheck
%type <defelt> opt_binary opt_oids copy_delimiter
%type <boolean> copy_from opt_program
@@ -10462,7 +10462,7 @@ cluster_index_specification:
*
*****************************************************************************/
-VacuumStmt: VACUUM opt_full opt_freeze opt_verbose opt_vacuum_relation_list
+VacuumStmt: VACUUM opt_full opt_freeze opt_verbose opt_analyze opt_vacuum_relation_list
{
VacuumStmt *n = makeNode(VacuumStmt);
n->options = VACOPT_VACUUM;
@@ -10472,19 +10472,9 @@ VacuumStmt: VACUUM opt_full opt_freeze opt_verbose opt_vacuum_relation_list
n->options |= VACOPT_FREEZE;
if ($4)
n->options |= VACOPT_VERBOSE;
- n->rels = $5;
- $$ = (Node *)n;
- }
- | VACUUM opt_full opt_freeze opt_verbose AnalyzeStmt
- {
- VacuumStmt *n = (VacuumStmt *) $5;
- n->options |= VACOPT_VACUUM;
- if ($2)
- n->options |= VACOPT_FULL;
- if ($3)
- n->options |= VACOPT_FREEZE;
- if ($4)
- n->options |= VACOPT_VERBOSE;
+ if ($5)
+ n->options |= VACOPT_ANALYZE;
+ n->rels = $6;
$$ = (Node *)n;
}
| VACUUM '(' vacuum_option_list ')' opt_vacuum_relation_list
@@ -10534,6 +10524,11 @@ analyze_keyword:
| ANALYSE /* British */ {}
;
+opt_analyze:
+ analyze_keyword { $$ = true; }
+ | /*EMPTY*/ { $$ = false; }
+ ;
+
opt_verbose:
VERBOSE { $$ = true; }
| /*EMPTY*/ { $$ = false; }
0002_add_parenthesized_analyzed_syntax_v2.patchapplication/octet-stream; name=0002_add_parenthesized_analyzed_syntax_v2.patchDownload
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 83b07a0..10b3a9a 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -21,9 +21,14 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
+ANALYZE [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
-<phrase>where <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
+<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
+
+ VERBOSE
+
+<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
<replaceable class="parameter">table_name</replaceable> [ ( <replaceable class="parameter">column_name</replaceable> [, ...] ) ]
</synopsis>
@@ -49,6 +54,13 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
It is further possible to give a list of column names for a table,
in which case only the statistics for those columns are collected.
</para>
+
+ <para>
+ When the option list is surrounded by parentheses, the options can be
+ written in any order. The parenthesized syntax was added in
+ <productname>PostgreSQL</productname> 11; the unparenthesized syntax
+ is deprecated.
+ </para>
</refsect1>
<refsect1>
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 3c76bda..d0a5203 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -306,6 +306,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> opt_lock lock_type cast_context
%type <ival> vacuum_option_list vacuum_option_elem
+ analyze_option_list analyze_option_elem
%type <boolean> opt_or_replace
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
@@ -10517,6 +10518,22 @@ AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
n->rels = $3;
$$ = (Node *)n;
}
+ | analyze_keyword '(' analyze_option_list ')' opt_vacuum_relation_list
+ {
+ VacuumStmt *n = makeNode(VacuumStmt);
+ n->options = VACOPT_ANALYZE | $3;
+ n->rels = $5;
+ $$ = (Node *) n;
+ }
+ ;
+
+analyze_option_list:
+ analyze_option_elem { $$ = $1; }
+ | analyze_option_list ',' analyze_option_elem { $$ = $1 | $3; }
+ ;
+
+analyze_option_elem:
+ VERBOSE { $$ = VACOPT_VERBOSE; }
;
analyze_keyword:
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index c440c7e..199dda6 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -112,6 +112,9 @@ ANALYZE vactst, does_not_exist, vacparted;
ERROR: relation "does_not_exist" does not exist
ANALYZE vactst (i), vacparted (does_not_exist);
ERROR: column "does_not_exist" of relation "vacparted" does not exist
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 92eaca2..e7b7a7f 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -89,6 +89,9 @@ ANALYZE vacparted (b), vactst;
ANALYZE vactst, does_not_exist, vacparted;
ANALYZE vactst (i), vacparted (does_not_exist);
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
0003_add_nowait_vacuum_option_v2.patchapplication/octet-stream; name=0003_add_nowait_vacuum_option_v2.patchDownload
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 10b3a9a..05882c3 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -27,6 +27,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
VERBOSE
+ NOWAIT
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -77,6 +78,17 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
</varlistentry>
<varlistentry>
+ <term><literal>NOWAIT</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>ANALYZE</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index b760e8e..1d7ffef 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -31,6 +31,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
VERBOSE
ANALYZE
DISABLE_PAGE_SKIPPING
+ NOWAIT
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -161,6 +162,17 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</varlistentry>
<varlistentry>
+ <term><literal>NOWAIT</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
<para>
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index f952b3c..67a5e58 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -159,13 +159,24 @@ analyze_rel(Oid relid, RangeVar *relation, int options,
*/
if (!onerel)
{
+ char *relname = NULL;
+
+ /*
+ * If VACOPT_NOWAIT is specified, we always try to emit a log
+ * statement if a relation is skipped, even if the provided RangeVar
+ * is NULL.
+ */
+ if (relation != NULL)
+ relname = relation->relname;
+ else if (!rel_lock)
+ relname = get_rel_name(relid);
+
/*
- * If the RangeVar is not defined, we do not have enough information
- * to provide a meaningful log statement. Chances are that
- * analyze_rel's caller has intentionally not provided this
- * information so that this logging is skipped, anyway.
+ * If relname is NULL, we do not have enough information to provide a
+ * meaningful log statement. Chances are that this information was
+ * intentionally not provided so that this logging is skipped, anyway.
*/
- if (relation == NULL)
+ if (relname == NULL)
return;
/*
@@ -185,12 +196,12 @@ analyze_rel(Oid relid, RangeVar *relation, int options,
ereport(elevel,
(errcode(ERRCODE_LOCK_NOT_AVAILABLE),
errmsg("skipping analyze of \"%s\" --- lock not available",
- relation->relname)));
+ relname)));
else
ereport(elevel,
(errcode(ERRCODE_UNDEFINED_TABLE),
errmsg("skipping analyze of \"%s\" --- relation no longer exists",
- relation->relname)));
+ relname)));
return;
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 4abe6b1..09a43d2 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -47,6 +47,7 @@
#include "utils/acl.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
+#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -1411,21 +1412,31 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
if (!onerel)
{
int elevel = 0;
+ char *relname = NULL;
+
+ /*
+ * If VACOPT_NOWAIT is specified, we always try to emit a log
+ * statement if a relation is skipped, even if the provided RangeVar
+ * is NULL.
+ */
+ if (relation != NULL)
+ relname = relation->relname;
+ else if (!rel_lock)
+ relname = get_rel_name(relid);
/*
* Determine the log level.
*
- * If the RangeVar is not defined, we do not have enough information
- * to provide a meaningful log statement. Chances are that
- * vacuum_rel's caller has intentionally not provided this information
- * so that this logging is skipped, anyway.
+ * If relname is NULL, we do not have enough information to provide a
+ * meaningful log statement. Chances are that this information was
+ * intentionally not provided so that this logging is skipped, anyway.
*
* Otherwise, for autovacuum logs, we emit a LOG if
* log_autovacuum_min_duration is not disabled. For manual VACUUM, we
* emit a WARNING to match the log statements in the permission
* checks.
*/
- if (relation != NULL)
+ if (relname != NULL)
{
if (!IsAutoVacuumWorkerProcess())
elevel = WARNING;
@@ -1439,12 +1450,12 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
ereport(elevel,
(errcode(ERRCODE_LOCK_NOT_AVAILABLE),
errmsg("skipping vacuum of \"%s\" --- lock not available",
- relation->relname)));
+ relname)));
else
ereport(elevel,
(errcode(ERRCODE_UNDEFINED_TABLE),
errmsg("skipping vacuum of \"%s\" --- relation no longer exists",
- relation->relname)));
+ relname)));
}
PopActiveSnapshot();
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index df6ce37..08c2548 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -10507,6 +10507,7 @@ vacuum_option_elem:
errmsg("unrecognized VACUUM option \"%s\"", $1),
parser_errposition(@1)));
}
+ | NOWAIT { $$ = VACOPT_NOWAIT; }
;
AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
@@ -10534,6 +10535,7 @@ analyze_option_list:
analyze_option_elem:
VERBOSE { $$ = VACOPT_VERBOSE; }
+ | NOWAIT { $$ = VACOPT_NOWAIT; }
;
analyze_keyword:
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2eaa6b2..27e06fb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3116,7 +3116,7 @@ typedef enum VacuumOption
VACOPT_VERBOSE = 1 << 2, /* print progress info */
VACOPT_FREEZE = 1 << 3, /* FREEZE option */
VACOPT_FULL = 1 << 4, /* FULL (non-concurrent) vacuum */
- VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock (autovacuum only) */
+ VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock */
VACOPT_SKIPTOAST = 1 << 6, /* don't process the TOAST table, if any */
VACOPT_DISABLE_PAGE_SKIPPING = 1 << 7 /* don't skip any pages */
} VacuumOption;
diff --git a/src/test/isolation/expected/vacuum-nowait.out b/src/test/isolation/expected/vacuum-nowait.out
new file mode 100644
index 0000000..a65f8be
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-nowait.out
@@ -0,0 +1,67 @@
+Parsed test spec with 2 sessions
+
+starting permutation: lock vac_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_specified: VACUUM (NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_all_parts: VACUUM (NOWAIT) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock analyze_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping analyze of "part1" --- lock not available
+step analyze_specified: ANALYZE (NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock analyze_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping analyze of "part1" --- lock not available
+step analyze_all_parts: ANALYZE (NOWAIT) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_analyze_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_analyze_specified: VACUUM (ANALYZE, NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_analyze_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_analyze_all_parts: VACUUM (ANALYZE, NOWAIT) parted;
+step commit:
+ COMMIT;
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index eb566eb..c1c09ea 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -64,3 +64,4 @@ test: async-notify
test: vacuum-reltuples
test: timeouts
test: vacuum-concurrent-drop
+test: vacuum-nowait
diff --git a/src/test/isolation/specs/vacuum-nowait.spec b/src/test/isolation/specs/vacuum-nowait.spec
new file mode 100644
index 0000000..ed27d4f
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-nowait.spec
@@ -0,0 +1,42 @@
+# Test for the NOWAIT option for VACUUM and ANALYZE commands.
+#
+# Skipped relations should be logged regardless of whether it was explicitly
+# specified in the command.
+
+setup
+{
+ CREATE TABLE parted (a INT) PARTITION BY LIST (a);
+ CREATE TABLE part1 PARTITION OF parted FOR VALUES IN (1);
+ CREATE TABLE part2 PARTITION OF parted FOR VALUES IN (2);
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS parted;
+}
+
+session "s1"
+step "lock"
+{
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+}
+step "commit"
+{
+ COMMIT;
+}
+
+session "s2"
+step "vac_specified" { VACUUM (NOWAIT) part1, part2; }
+step "vac_all_parts" { VACUUM (NOWAIT) parted; }
+step "analyze_specified" { ANALYZE (NOWAIT) part1, part2; }
+step "analyze_all_parts" { ANALYZE (NOWAIT) parted; }
+step "vac_analyze_specified" { VACUUM (ANALYZE, NOWAIT) part1, part2; }
+step "vac_analyze_all_parts" { VACUUM (ANALYZE, NOWAIT) parted; }
+
+permutation "lock" "vac_specified" "commit"
+permutation "lock" "vac_all_parts" "commit"
+permutation "lock" "analyze_specified" "commit"
+permutation "lock" "analyze_all_parts" "commit"
+permutation "lock" "vac_analyze_specified" "commit"
+permutation "lock" "vac_analyze_all_parts" "commit"
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index 199dda6..99aad21 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -113,8 +113,11 @@ ERROR: relation "does_not_exist" does not exist
ANALYZE vactst (i), vacparted (does_not_exist);
ERROR: column "does_not_exist" of relation "vacparted" does not exist
-- parenthesized syntax for ANALYZE
-ANALYZE (VERBOSE) does_not_exist;
+ANALYZE (NOWAIT, VERBOSE) does_not_exist;
ERROR: relation "does_not_exist" does not exist
+-- nowait option
+VACUUM (NOWAIT) vactst;
+ANALYZE (NOWAIT) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index e7b7a7f..5e88c68 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -90,7 +90,11 @@ ANALYZE vactst, does_not_exist, vacparted;
ANALYZE vactst (i), vacparted (does_not_exist);
-- parenthesized syntax for ANALYZE
-ANALYZE (VERBOSE) does_not_exist;
+ANALYZE (NOWAIT, VERBOSE) does_not_exist;
+
+-- nowait option
+VACUUM (NOWAIT) vactst;
+ANALYZE (NOWAIT) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
On Tue, Dec 19, 2017 at 2:35 AM, Bossart, Nathan <bossartn@amazon.com> wrote:
On 12/11/17, 9:41 PM, "Masahiko Sawada" <sawada.mshk@gmail.com> wrote:
For 0003 patch, if we do VACUUM (NOWAIT) to whole database we don't
get the above WARNING messages, but I think we should get them. The
reported issue also did VACUUM FULL and REINDEX to whole database.
Especially when VACUUM to whole database the information of skipped
relations would be useful for users.Here is a new set of patches. 0001 and 0002 are essentially the same
as before except for a rebase and some small indentation fixes. 0003
now includes logic to log all skipped relations regardless of whether
they were specified in the command. I've also modified the isolation
test slightly to use partitioned tables instead of running VACUUM and
ANALYZE on the whole database. This helps prevent timeouts on build-
farm machines with CLOBBER_CACHE_ALWAYS enabled (like with 0a3edbb3).
Thank you for updating the patches.
According to the following old comment, there might be reason why we
didn't pass the information to vacuum_rel(). But your patch fetches
the relation
name even if the "relation" is not provided. I wonder if it can be
problem in some cases.
- * If the RangeVar is not defined, we do not have
enough information
- * to provide a meaningful log statement. Chances are that
- * vacuum_rel's caller has intentionally not provided
this information
- * so that this logging is skipped, anyway.
+ * If relname is NULL, we do not have enough
information to provide a
+ * meaningful log statement. Chances are that this
information was
+ * intentionally not provided so that this logging is
skipped, anyway.
It would be an another discussion but autovacuum logs usually use
explicit names. Since the following two logs could be emitted by
autovacuum I wonder if we can make an explicit relation name if we're
autovacuum.
if (elevel != 0)
{
if (!rel_lock)
ereport(elevel,
(errcode(ERRCODE_LOCK_NOT_AVAILABLE),
errmsg("skipping vacuum of \"%s\" --- lock not available",
relname)));
else
ereport(elevel,
(errcode(ERRCODE_UNDEFINED_TABLE),
errmsg("skipping vacuum of \"%s\" --- relation
no longer exists",
relname)));
}
Regards,
--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On 12/18/17, 3:30 PM, "Masahiko Sawada" <sawada.mshk@gmail.com> wrote:
According to the following old comment, there might be reason why we
didn't pass the information to vacuum_rel(). But your patch fetches
the relation
name even if the "relation" is not provided. I wonder if it can be
problem in some cases.
Thanks for taking another look.
I've thought through a few edge cases here, but I haven't noticed
anything that I think is a problem. If an unspecified relation is
renamed prior to get_rel_name(), we'll use the updated name, which
doesn't seem like an issue. If an unspecified relation is renamed
between get_rel_name() and the log statement, we'll use the old name,
which seems possible in the current logging logic for VACUUM/ANALYZE.
And if an unspecified relation is dropped just prior to
get_rel_name(), the result will be NULL, and the logging will be
skipped (as it already is for concurrently dropped relations that are
not specified in the command).
Nathan
On Tue, Dec 19, 2017 at 7:18 AM, Bossart, Nathan <bossartn@amazon.com> wrote:
On 12/18/17, 3:30 PM, "Masahiko Sawada" <sawada.mshk@gmail.com> wrote:
According to the following old comment, there might be reason why we
didn't pass the information to vacuum_rel(). But your patch fetches
the relation
name even if the "relation" is not provided. I wonder if it can be
problem in some cases.Thanks for taking another look.
I've thought through a few edge cases here, but I haven't noticed
anything that I think is a problem. If an unspecified relation is
renamed prior to get_rel_name(), we'll use the updated name, which
doesn't seem like an issue. If an unspecified relation is renamed
between get_rel_name() and the log statement, we'll use the old name,
which seems possible in the current logging logic for VACUUM/ANALYZE.
And if an unspecified relation is dropped just prior to
get_rel_name(), the result will be NULL, and the logging will be
skipped (as it already is for concurrently dropped relations that are
not specified in the command).
Thank you for explanation.
There are three cases where "relation" is set NULL:
* Vacuum to whole database
* Child tables when vacuum to parent table
* TOAST tables when vacuum to normal table
As current related comment says, it would not be appropriate to
complain if we could not open such unspecified relations later. But
with you patch, we would complain it only if we could not take a lock
on these relations. It's fine with me.
On the latest patch, it looks good to me except for a following comment.
- * If the RangeVar is not defined, we do not have
enough information
- * to provide a meaningful log statement. Chances are that
- * vacuum_rel's caller has intentionally not provided
this information
- * so that this logging is skipped, anyway.
+ * If relname is NULL, we do not have enough
information to provide a
+ * meaningful log statement. Chances are that this
information was
+ * intentionally not provided so that this logging is
skipped, anyway.
This comment looks odd to me because we ourselves didn't set relname
just before. For example, we can move the sentence to above comment
block as follows. Thoughts?
/*
* If relation is NULL, we do not have enough information to provide a
* meaningful log statement. Chances are that this information was
* intentionally not provided so that this logging is skipped, anyway.
* However, iff VACOPT_NOWAIT is specified, we always try to emit
* a log statement even if relation is NULL.
*/
(snip)
/*
* Determine the log level.
*
* For autovacuum logs, we emit a LOG if
* log_autovacuum_min_duration is not diabled. For manual VACUUM, we
* emit a WARNING to match the log statements in the permission
* checks.
*/
Regards,
--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On 12/21/17, 11:07 PM, "Masahiko Sawada" <sawada.mshk@gmail.com> wrote:
On the latest patch, it looks good to me except for a following comment.
- * If the RangeVar is not defined, we do not have enough information - * to provide a meaningful log statement. Chances are that - * vacuum_rel's caller has intentionally not provided this information - * so that this logging is skipped, anyway. + * If relname is NULL, we do not have enough information to provide a + * meaningful log statement. Chances are that this information was + * intentionally not provided so that this logging is skipped, anyway.This comment looks odd to me because we ourselves didn't set relname
just before. For example, we can move the sentence to above comment
block as follows. Thoughts?/*
* If relation is NULL, we do not have enough information to provide a
* meaningful log statement. Chances are that this information was
* intentionally not provided so that this logging is skipped, anyway.
* However, iff VACOPT_NOWAIT is specified, we always try to emit
* a log statement even if relation is NULL.
*/(snip)
/*
* Determine the log level.
*
* For autovacuum logs, we emit a LOG if
* log_autovacuum_min_duration is not diabled. For manual VACUUM, we
* emit a WARNING to match the log statements in the permission
* checks.
*/
I agree, this makes more sense. I've made this change in v3 of 0003.
Nathan
Attachments:
0003_add_nowait_vacuum_option_v3.patchapplication/octet-stream; name=0003_add_nowait_vacuum_option_v3.patchDownload
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 10b3a9a..05882c3 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -27,6 +27,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
VERBOSE
+ NOWAIT
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -77,6 +78,17 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
</varlistentry>
<varlistentry>
+ <term><literal>NOWAIT</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>ANALYZE</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index b760e8e..1d7ffef 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -31,6 +31,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
VERBOSE
ANALYZE
DISABLE_PAGE_SKIPPING
+ NOWAIT
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -161,6 +162,17 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</varlistentry>
<varlistentry>
+ <term><literal>NOWAIT</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
<para>
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index f952b3c..e3e2444 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -159,13 +159,22 @@ analyze_rel(Oid relid, RangeVar *relation, int options,
*/
if (!onerel)
{
+ char *relname = NULL;
+
/*
- * If the RangeVar is not defined, we do not have enough information
- * to provide a meaningful log statement. Chances are that
- * analyze_rel's caller has intentionally not provided this
- * information so that this logging is skipped, anyway.
+ * If relation is NULL, we do not have enough information to provide a
+ * meaningful log statement. Chances are that this information was
+ * intentionally not provided so that this logging is skipped, anyway.
+ * However, if VACOPT_NOWAIT is specified, we always try to emit a log
+ * statement if a relation is skipped, even if the provided RangeVar
+ * is NULL.
*/
- if (relation == NULL)
+ if (relation != NULL)
+ relname = relation->relname;
+ else if (!rel_lock)
+ relname = get_rel_name(relid);
+
+ if (relname == NULL)
return;
/*
@@ -185,12 +194,12 @@ analyze_rel(Oid relid, RangeVar *relation, int options,
ereport(elevel,
(errcode(ERRCODE_LOCK_NOT_AVAILABLE),
errmsg("skipping analyze of \"%s\" --- lock not available",
- relation->relname)));
+ relname)));
else
ereport(elevel,
(errcode(ERRCODE_UNDEFINED_TABLE),
errmsg("skipping analyze of \"%s\" --- relation no longer exists",
- relation->relname)));
+ relname)));
return;
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 4abe6b1..60ac885 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -47,6 +47,7 @@
#include "utils/acl.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
+#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -1411,21 +1412,29 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
if (!onerel)
{
int elevel = 0;
+ char *relname = NULL;
+
+ /*
+ * If relation is NULL, we do not have enough information to provide a
+ * meaningful log statement. Chances are that this information was
+ * intentionally not provided so that this logging is skipped, anyway.
+ * However, if VACOPT_NOWAIT is specified, we always try to emit a log
+ * statement if a relation is skipped, even if the provided RangeVar
+ * is NULL.
+ */
+ if (relation != NULL)
+ relname = relation->relname;
+ else if (!rel_lock)
+ relname = get_rel_name(relid);
/*
* Determine the log level.
*
- * If the RangeVar is not defined, we do not have enough information
- * to provide a meaningful log statement. Chances are that
- * vacuum_rel's caller has intentionally not provided this information
- * so that this logging is skipped, anyway.
- *
- * Otherwise, for autovacuum logs, we emit a LOG if
- * log_autovacuum_min_duration is not disabled. For manual VACUUM, we
- * emit a WARNING to match the log statements in the permission
- * checks.
+ * For autovacuum logs, we emit a LOG if log_autovacuum_min_duration
+ * is not disabled. For manual VACUUM, we emit a WARNING to match the
+ * log statements in the permission checks.
*/
- if (relation != NULL)
+ if (relname != NULL)
{
if (!IsAutoVacuumWorkerProcess())
elevel = WARNING;
@@ -1439,12 +1448,12 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
ereport(elevel,
(errcode(ERRCODE_LOCK_NOT_AVAILABLE),
errmsg("skipping vacuum of \"%s\" --- lock not available",
- relation->relname)));
+ relname)));
else
ereport(elevel,
(errcode(ERRCODE_UNDEFINED_TABLE),
errmsg("skipping vacuum of \"%s\" --- relation no longer exists",
- relation->relname)));
+ relname)));
}
PopActiveSnapshot();
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d0a5203..f7d5fb5 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -10507,6 +10507,7 @@ vacuum_option_elem:
errmsg("unrecognized VACUUM option \"%s\"", $1),
parser_errposition(@1)));
}
+ | NOWAIT { $$ = VACOPT_NOWAIT; }
;
AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
@@ -10534,6 +10535,7 @@ analyze_option_list:
analyze_option_elem:
VERBOSE { $$ = VACOPT_VERBOSE; }
+ | NOWAIT { $$ = VACOPT_NOWAIT; }
;
analyze_keyword:
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2eaa6b2..27e06fb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3116,7 +3116,7 @@ typedef enum VacuumOption
VACOPT_VERBOSE = 1 << 2, /* print progress info */
VACOPT_FREEZE = 1 << 3, /* FREEZE option */
VACOPT_FULL = 1 << 4, /* FULL (non-concurrent) vacuum */
- VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock (autovacuum only) */
+ VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock */
VACOPT_SKIPTOAST = 1 << 6, /* don't process the TOAST table, if any */
VACOPT_DISABLE_PAGE_SKIPPING = 1 << 7 /* don't skip any pages */
} VacuumOption;
diff --git a/src/test/isolation/expected/vacuum-nowait.out b/src/test/isolation/expected/vacuum-nowait.out
new file mode 100644
index 0000000..a65f8be
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-nowait.out
@@ -0,0 +1,67 @@
+Parsed test spec with 2 sessions
+
+starting permutation: lock vac_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_specified: VACUUM (NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_all_parts: VACUUM (NOWAIT) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock analyze_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping analyze of "part1" --- lock not available
+step analyze_specified: ANALYZE (NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock analyze_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping analyze of "part1" --- lock not available
+step analyze_all_parts: ANALYZE (NOWAIT) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_analyze_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_analyze_specified: VACUUM (ANALYZE, NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_analyze_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_analyze_all_parts: VACUUM (ANALYZE, NOWAIT) parted;
+step commit:
+ COMMIT;
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index eb566eb..c1c09ea 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -64,3 +64,4 @@ test: async-notify
test: vacuum-reltuples
test: timeouts
test: vacuum-concurrent-drop
+test: vacuum-nowait
diff --git a/src/test/isolation/specs/vacuum-nowait.spec b/src/test/isolation/specs/vacuum-nowait.spec
new file mode 100644
index 0000000..ed27d4f
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-nowait.spec
@@ -0,0 +1,42 @@
+# Test for the NOWAIT option for VACUUM and ANALYZE commands.
+#
+# Skipped relations should be logged regardless of whether it was explicitly
+# specified in the command.
+
+setup
+{
+ CREATE TABLE parted (a INT) PARTITION BY LIST (a);
+ CREATE TABLE part1 PARTITION OF parted FOR VALUES IN (1);
+ CREATE TABLE part2 PARTITION OF parted FOR VALUES IN (2);
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS parted;
+}
+
+session "s1"
+step "lock"
+{
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+}
+step "commit"
+{
+ COMMIT;
+}
+
+session "s2"
+step "vac_specified" { VACUUM (NOWAIT) part1, part2; }
+step "vac_all_parts" { VACUUM (NOWAIT) parted; }
+step "analyze_specified" { ANALYZE (NOWAIT) part1, part2; }
+step "analyze_all_parts" { ANALYZE (NOWAIT) parted; }
+step "vac_analyze_specified" { VACUUM (ANALYZE, NOWAIT) part1, part2; }
+step "vac_analyze_all_parts" { VACUUM (ANALYZE, NOWAIT) parted; }
+
+permutation "lock" "vac_specified" "commit"
+permutation "lock" "vac_all_parts" "commit"
+permutation "lock" "analyze_specified" "commit"
+permutation "lock" "analyze_all_parts" "commit"
+permutation "lock" "vac_analyze_specified" "commit"
+permutation "lock" "vac_analyze_all_parts" "commit"
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index 199dda6..99aad21 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -113,8 +113,11 @@ ERROR: relation "does_not_exist" does not exist
ANALYZE vactst (i), vacparted (does_not_exist);
ERROR: column "does_not_exist" of relation "vacparted" does not exist
-- parenthesized syntax for ANALYZE
-ANALYZE (VERBOSE) does_not_exist;
+ANALYZE (NOWAIT, VERBOSE) does_not_exist;
ERROR: relation "does_not_exist" does not exist
+-- nowait option
+VACUUM (NOWAIT) vactst;
+ANALYZE (NOWAIT) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index e7b7a7f..5e88c68 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -90,7 +90,11 @@ ANALYZE vactst, does_not_exist, vacparted;
ANALYZE vactst (i), vacparted (does_not_exist);
-- parenthesized syntax for ANALYZE
-ANALYZE (VERBOSE) does_not_exist;
+ANALYZE (NOWAIT, VERBOSE) does_not_exist;
+
+-- nowait option
+VACUUM (NOWAIT) vactst;
+ANALYZE (NOWAIT) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
0001_fix_unparenthesized_vacuum_grammar_v2.patchapplication/octet-stream; name=0001_fix_unparenthesized_vacuum_grammar_v2.patchDownload
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index ebfc94f..3c76bda 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -437,7 +437,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <boolean> opt_instead
%type <boolean> opt_unique opt_concurrently opt_verbose opt_full
-%type <boolean> opt_freeze opt_default opt_recheck
+%type <boolean> opt_freeze opt_analyze opt_default opt_recheck
%type <defelt> opt_binary opt_oids copy_delimiter
%type <boolean> copy_from opt_program
@@ -10462,7 +10462,7 @@ cluster_index_specification:
*
*****************************************************************************/
-VacuumStmt: VACUUM opt_full opt_freeze opt_verbose opt_vacuum_relation_list
+VacuumStmt: VACUUM opt_full opt_freeze opt_verbose opt_analyze opt_vacuum_relation_list
{
VacuumStmt *n = makeNode(VacuumStmt);
n->options = VACOPT_VACUUM;
@@ -10472,19 +10472,9 @@ VacuumStmt: VACUUM opt_full opt_freeze opt_verbose opt_vacuum_relation_list
n->options |= VACOPT_FREEZE;
if ($4)
n->options |= VACOPT_VERBOSE;
- n->rels = $5;
- $$ = (Node *)n;
- }
- | VACUUM opt_full opt_freeze opt_verbose AnalyzeStmt
- {
- VacuumStmt *n = (VacuumStmt *) $5;
- n->options |= VACOPT_VACUUM;
- if ($2)
- n->options |= VACOPT_FULL;
- if ($3)
- n->options |= VACOPT_FREEZE;
- if ($4)
- n->options |= VACOPT_VERBOSE;
+ if ($5)
+ n->options |= VACOPT_ANALYZE;
+ n->rels = $6;
$$ = (Node *)n;
}
| VACUUM '(' vacuum_option_list ')' opt_vacuum_relation_list
@@ -10534,6 +10524,11 @@ analyze_keyword:
| ANALYSE /* British */ {}
;
+opt_analyze:
+ analyze_keyword { $$ = true; }
+ | /*EMPTY*/ { $$ = false; }
+ ;
+
opt_verbose:
VERBOSE { $$ = true; }
| /*EMPTY*/ { $$ = false; }
0002_add_parenthesized_analyzed_syntax_v2.patchapplication/octet-stream; name=0002_add_parenthesized_analyzed_syntax_v2.patchDownload
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 83b07a0..10b3a9a 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -21,9 +21,14 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
+ANALYZE [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
-<phrase>where <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
+<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
+
+ VERBOSE
+
+<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
<replaceable class="parameter">table_name</replaceable> [ ( <replaceable class="parameter">column_name</replaceable> [, ...] ) ]
</synopsis>
@@ -49,6 +54,13 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
It is further possible to give a list of column names for a table,
in which case only the statistics for those columns are collected.
</para>
+
+ <para>
+ When the option list is surrounded by parentheses, the options can be
+ written in any order. The parenthesized syntax was added in
+ <productname>PostgreSQL</productname> 11; the unparenthesized syntax
+ is deprecated.
+ </para>
</refsect1>
<refsect1>
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 3c76bda..d0a5203 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -306,6 +306,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> opt_lock lock_type cast_context
%type <ival> vacuum_option_list vacuum_option_elem
+ analyze_option_list analyze_option_elem
%type <boolean> opt_or_replace
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
@@ -10517,6 +10518,22 @@ AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
n->rels = $3;
$$ = (Node *)n;
}
+ | analyze_keyword '(' analyze_option_list ')' opt_vacuum_relation_list
+ {
+ VacuumStmt *n = makeNode(VacuumStmt);
+ n->options = VACOPT_ANALYZE | $3;
+ n->rels = $5;
+ $$ = (Node *) n;
+ }
+ ;
+
+analyze_option_list:
+ analyze_option_elem { $$ = $1; }
+ | analyze_option_list ',' analyze_option_elem { $$ = $1 | $3; }
+ ;
+
+analyze_option_elem:
+ VERBOSE { $$ = VACOPT_VERBOSE; }
;
analyze_keyword:
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index c440c7e..199dda6 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -112,6 +112,9 @@ ANALYZE vactst, does_not_exist, vacparted;
ERROR: relation "does_not_exist" does not exist
ANALYZE vactst (i), vacparted (does_not_exist);
ERROR: column "does_not_exist" of relation "vacparted" does not exist
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 92eaca2..e7b7a7f 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -89,6 +89,9 @@ ANALYZE vacparted (b), vactst;
ANALYZE vactst, does_not_exist, vacparted;
ANALYZE vactst (i), vacparted (does_not_exist);
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
On Thu, Dec 28, 2017 at 10:46:18PM +0000, Bossart, Nathan wrote:
I agree, this makes more sense. I've made this change in v3 of 0003.
Based on the opinions gathered on this thread, 0001 and 0002 seem to be
in the shape wanted, and those look good for shipping. Now for 0003 we
are not there yet.
- if (relation == NULL)
+ if (relation != NULL)
+ relname = relation->relname;
+ else if (!rel_lock)
+ relname = get_rel_name(relid);
+
+ if (relname == NULL)
I think that you are doing it wrong here. In get_all_vacuum_rels() you
should build a RangeVar to be reused in the context of this error
message, and hence you'll save an extra lookup based on the relation
OID here, saving from any timing issues that you have overseen as in
this code path a lock on the relation whose name is looked at is not
taken. Relying on the RangeVar being NULL to not generate any logs is
fine as a concept to me, but let's fill it where it is needed, and in
the case of this patch a VACUUM NOWAIT on the whole database is such a
case.
--
Michael
On Mon, Jan 8, 2018 at 11:27 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Thu, Dec 28, 2017 at 10:46:18PM +0000, Bossart, Nathan wrote:
I agree, this makes more sense. I've made this change in v3 of 0003.
Based on the opinions gathered on this thread, 0001 and 0002 seem to be
in the shape wanted, and those look good for shipping.
Committed 0001.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 1/8/18, 10:28 PM, "Michael Paquier" <michael.paquier@gmail.com> wrote:
Based on the opinions gathered on this thread, 0001 and 0002 seem to be
in the shape wanted, and those look good for shipping. Now for 0003 we
are not there yet.
Thanks for taking a look.
- if (relation == NULL) + if (relation != NULL) + relname = relation->relname; + else if (!rel_lock) + relname = get_rel_name(relid); + + if (relname == NULL) I think that you are doing it wrong here. In get_all_vacuum_rels() you should build a RangeVar to be reused in the context of this error message, and hence you'll save an extra lookup based on the relation OID here, saving from any timing issues that you have overseen as in this code path a lock on the relation whose name is looked at is not taken. Relying on the RangeVar being NULL to not generate any logs is fine as a concept to me, but let's fill it where it is needed, and in the case of this patch a VACUUM NOWAIT on the whole database is such a case.
I understand what you are saying here. I think there are two competing
logging behaviors:
1. If a relation is concurrently dropped, we should skip logging if
the relation was not specified in the original VACUUM/ANALYZE
command [0]/messages/by-id/28748.1507071576@sss.pgh.pa.us
2. If a relation is skipped because the lock is not available, we
should never skip logging
Since we rely on the RangeVar being NULL to determine whether or not
we should log for case 1, filling in the RangeVar during
get_all_vacuum_rels() would add complexity. Any time VACOPT_NOWAIT is
specified, we must construct a RangeVar for all relations. Also, we
must find a new way to track whether or not the relation was specified
in the original command in order to maintain the behavior of case 1.
IMO it is simpler to call get_rel_name() to discover the relation name
in this specific case (NOWAIT is specified, lock is not available, and
the relation was not specified in the original command). I detailed
some potential edge cases of this approach earlier [1]/messages/by-id/32F4B778-292B-41AF-891A-497B8127CD59@amazon.com. Even if I am
missing some, I think the worst case is that get_rel_name() returns
NULL and the logging is skipped.
What do you think? I will take a deeper look into how your suggested
approach might be achieved.
Nathan
[0]: /messages/by-id/28748.1507071576@sss.pgh.pa.us
[1]: /messages/by-id/32F4B778-292B-41AF-891A-497B8127CD59@amazon.com
On 1/9/18, 9:24 AM, "Robert Haas" <robertmhaas@gmail.com> wrote:
On Mon, Jan 8, 2018 at 11:27 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Dec 28, 2017 at 10:46:18PM +0000, Bossart, Nathan wrote:
I agree, this makes more sense. I've made this change in v3 of 0003.
Based on the opinions gathered on this thread, 0001 and 0002 seem to be
in the shape wanted, and those look good for shipping.Committed 0001.
Thanks!
Nathan
On 2018-01-09 10:23:12 -0500, Robert Haas wrote:
On Mon, Jan 8, 2018 at 11:27 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Dec 28, 2017 at 10:46:18PM +0000, Bossart, Nathan wrote:
I agree, this makes more sense. I've made this change in v3 of 0003.
Based on the opinions gathered on this thread, 0001 and 0002 seem to be
in the shape wanted, and those look good for shipping.Committed 0001.
FWIW, I think in the future it'd be good to develop new features outside
of a thread about "vacuum crashes".
Greetings,
Andres Freund
On Tue, Jan 09, 2018 at 09:40:50PM +0000, Bossart, Nathan wrote:
On 1/8/18, 10:28 PM, "Michael Paquier" <michael.paquier@gmail.com> wrote:
I think that you are doing it wrong here. In get_all_vacuum_rels() you
should build a RangeVar to be reused in the context of this error
message, and hence you'll save an extra lookup based on the relation
OID here, saving from any timing issues that you have overseen as in
this code path a lock on the relation whose name is looked at is not
taken. Relying on the RangeVar being NULL to not generate any logs is
fine as a concept to me, but let's fill it where it is needed, and in
the case of this patch a VACUUM NOWAIT on the whole database is such a
case.I understand what you are saying here. I think there are two competing
logging behaviors:1. If a relation is concurrently dropped, we should skip logging if
the relation was not specified in the original VACUUM/ANALYZE
command [0]
2. If a relation is skipped because the lock is not available, we
should never skip logging
At the end this comes back to if the relation is explicitely listed or
not in the command specified by the user..
What do you think? I will take a deeper look into how your suggested
approach might be achieved.
Backpedalling a bit on this point and coming back to this message from
Tom (/messages/by-id/28748.1507071576@sss.pgh.pa.us)
which you just cited. Why do we actually need to issue any WARNING
messages for unlisted relations? Contrary to what Sawada-san complained
upthread, it looks sane to me to not log anything if a relation is not
explicitely listed. So you should not get any warnings for a
database-wide VACUUM if a relation is dropped while the thing is
running, and what you proposed initially in
/messages/by-id/D3FC73E2-9B1A-4DB4-8180-55F57D116B4E@amazon.com
is more simple, does not need to worry about any kind of timing issues,
and is consistent with autovacuum.
--
Michael
On Tue, Jan 09, 2018 at 01:46:55PM -0800, Andres Freund wrote:
On 2018-01-09 10:23:12 -0500, Robert Haas wrote:
On Mon, Jan 8, 2018 at 11:27 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Dec 28, 2017 at 10:46:18PM +0000, Bossart, Nathan wrote:
I agree, this makes more sense. I've made this change in v3 of 0003.
Based on the opinions gathered on this thread, 0001 and 0002 seem to be
in the shape wanted, and those look good for shipping.Committed 0001.
FWIW, I think in the future it'd be good to develop new features outside
of a thread about "vacuum crashes".
You are right atht this is bad practice. Partially my fault for not
recommending the move to a new thread.
--
Michael
On 1/9/18, 8:54 PM, "Michael Paquier" <michael.paquier@gmail.com> wrote:
Backpedalling a bit on this point and coming back to this message from
Tom (/messages/by-id/28748.1507071576@sss.pgh.pa.us)
which you just cited. Why do we actually need to issue any WARNING
messages for unlisted relations? Contrary to what Sawada-san complained
upthread, it looks sane to me to not log anything if a relation is not
explicitely listed. So you should not get any warnings for a
database-wide VACUUM if a relation is dropped while the thing is
running, and what you proposed initially in
/messages/by-id/D3FC73E2-9B1A-4DB4-8180-55F57D116B4E@amazon.com
is more simple, does not need to worry about any kind of timing issues,
and is consistent with autovacuum.
Right. I don't have a terribly strong opinion either way. I think the
counter-argument is that logging skipped relations might provide
valuable feedback to users. If I ran a database-wide VACUUM and a
relation was skipped due to lock contention, it might be nice to know
that so I can handle the relation individually.
Perhaps any logging changes for VACOPT_NOWAIT could be handled in a
separate thread. For now, I've updated 0003 to remove the logging
changes.
Nathan
Attachments:
0003_add_nowait_vacuum_option_v4.patchapplication/octet-stream; name=0003_add_nowait_vacuum_option_v4.patchDownload
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 10b3a9a..05882c3 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -27,6 +27,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
VERBOSE
+ NOWAIT
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -77,6 +78,17 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
</varlistentry>
<varlistentry>
+ <term><literal>NOWAIT</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>ANALYZE</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index b760e8e..1d7ffef 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -31,6 +31,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
VERBOSE
ANALYZE
DISABLE_PAGE_SKIPPING
+ NOWAIT
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -161,6 +162,17 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</varlistentry>
<varlistentry>
+ <term><literal>NOWAIT</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
<para>
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 1c8f115..34cf156 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -10507,6 +10507,7 @@ vacuum_option_elem:
errmsg("unrecognized VACUUM option \"%s\"", $1),
parser_errposition(@1)));
}
+ | NOWAIT { $$ = VACOPT_NOWAIT; }
;
AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
@@ -10534,6 +10535,7 @@ analyze_option_list:
analyze_option_elem:
VERBOSE { $$ = VACOPT_VERBOSE; }
+ | NOWAIT { $$ = VACOPT_NOWAIT; }
;
analyze_keyword:
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index b72178e..2d0f581 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3116,7 +3116,7 @@ typedef enum VacuumOption
VACOPT_VERBOSE = 1 << 2, /* print progress info */
VACOPT_FREEZE = 1 << 3, /* FREEZE option */
VACOPT_FULL = 1 << 4, /* FULL (non-concurrent) vacuum */
- VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock (autovacuum only) */
+ VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock */
VACOPT_SKIPTOAST = 1 << 6, /* don't process the TOAST table, if any */
VACOPT_DISABLE_PAGE_SKIPPING = 1 << 7 /* don't skip any pages */
} VacuumOption;
diff --git a/src/test/isolation/expected/vacuum-nowait.out b/src/test/isolation/expected/vacuum-nowait.out
new file mode 100644
index 0000000..225ec23
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-nowait.out
@@ -0,0 +1,64 @@
+Parsed test spec with 2 sessions
+
+starting permutation: lock vac_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_specified: VACUUM (NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step vac_all_parts: VACUUM (NOWAIT) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock analyze_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping analyze of "part1" --- lock not available
+step analyze_specified: ANALYZE (NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock analyze_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step analyze_all_parts: ANALYZE (NOWAIT) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_analyze_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_analyze_specified: VACUUM (ANALYZE, NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_analyze_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step vac_analyze_all_parts: VACUUM (ANALYZE, NOWAIT) parted;
+step commit:
+ COMMIT;
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index befe676..9d0dc7b 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -65,3 +65,4 @@ test: async-notify
test: vacuum-reltuples
test: timeouts
test: vacuum-concurrent-drop
+test: vacuum-nowait
diff --git a/src/test/isolation/specs/vacuum-nowait.spec b/src/test/isolation/specs/vacuum-nowait.spec
new file mode 100644
index 0000000..498b0a0
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-nowait.spec
@@ -0,0 +1,42 @@
+# Test for the NOWAIT option for VACUUM and ANALYZE commands.
+#
+# This also verifies that log messages are not emitted for skipped relations
+# that were not specified in the VACUUM or ANALYZE command.
+
+setup
+{
+ CREATE TABLE parted (a INT) PARTITION BY LIST (a);
+ CREATE TABLE part1 PARTITION OF parted FOR VALUES IN (1);
+ CREATE TABLE part2 PARTITION OF parted FOR VALUES IN (2);
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS parted;
+}
+
+session "s1"
+step "lock"
+{
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+}
+step "commit"
+{
+ COMMIT;
+}
+
+session "s2"
+step "vac_specified" { VACUUM (NOWAIT) part1, part2; }
+step "vac_all_parts" { VACUUM (NOWAIT) parted; }
+step "analyze_specified" { ANALYZE (NOWAIT) part1, part2; }
+step "analyze_all_parts" { ANALYZE (NOWAIT) parted; }
+step "vac_analyze_specified" { VACUUM (ANALYZE, NOWAIT) part1, part2; }
+step "vac_analyze_all_parts" { VACUUM (ANALYZE, NOWAIT) parted; }
+
+permutation "lock" "vac_specified" "commit"
+permutation "lock" "vac_all_parts" "commit"
+permutation "lock" "analyze_specified" "commit"
+permutation "lock" "analyze_all_parts" "commit"
+permutation "lock" "vac_analyze_specified" "commit"
+permutation "lock" "vac_analyze_all_parts" "commit"
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index 199dda6..99aad21 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -113,8 +113,11 @@ ERROR: relation "does_not_exist" does not exist
ANALYZE vactst (i), vacparted (does_not_exist);
ERROR: column "does_not_exist" of relation "vacparted" does not exist
-- parenthesized syntax for ANALYZE
-ANALYZE (VERBOSE) does_not_exist;
+ANALYZE (NOWAIT, VERBOSE) does_not_exist;
ERROR: relation "does_not_exist" does not exist
+-- nowait option
+VACUUM (NOWAIT) vactst;
+ANALYZE (NOWAIT) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index e7b7a7f..5e88c68 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -90,7 +90,11 @@ ANALYZE vactst, does_not_exist, vacparted;
ANALYZE vactst (i), vacparted (does_not_exist);
-- parenthesized syntax for ANALYZE
-ANALYZE (VERBOSE) does_not_exist;
+ANALYZE (NOWAIT, VERBOSE) does_not_exist;
+
+-- nowait option
+VACUUM (NOWAIT) vactst;
+ANALYZE (NOWAIT) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
0002_add_parenthesized_analyzed_syntax_v2.patchapplication/octet-stream; name=0002_add_parenthesized_analyzed_syntax_v2.patchDownload
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 83b07a0..10b3a9a 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -21,9 +21,14 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
+ANALYZE [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
-<phrase>where <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
+<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
+
+ VERBOSE
+
+<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
<replaceable class="parameter">table_name</replaceable> [ ( <replaceable class="parameter">column_name</replaceable> [, ...] ) ]
</synopsis>
@@ -49,6 +54,13 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
It is further possible to give a list of column names for a table,
in which case only the statistics for those columns are collected.
</para>
+
+ <para>
+ When the option list is surrounded by parentheses, the options can be
+ written in any order. The parenthesized syntax was added in
+ <productname>PostgreSQL</productname> 11; the unparenthesized syntax
+ is deprecated.
+ </para>
</refsect1>
<refsect1>
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 3c76bda..d0a5203 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -306,6 +306,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> opt_lock lock_type cast_context
%type <ival> vacuum_option_list vacuum_option_elem
+ analyze_option_list analyze_option_elem
%type <boolean> opt_or_replace
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
@@ -10517,6 +10518,22 @@ AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
n->rels = $3;
$$ = (Node *)n;
}
+ | analyze_keyword '(' analyze_option_list ')' opt_vacuum_relation_list
+ {
+ VacuumStmt *n = makeNode(VacuumStmt);
+ n->options = VACOPT_ANALYZE | $3;
+ n->rels = $5;
+ $$ = (Node *) n;
+ }
+ ;
+
+analyze_option_list:
+ analyze_option_elem { $$ = $1; }
+ | analyze_option_list ',' analyze_option_elem { $$ = $1 | $3; }
+ ;
+
+analyze_option_elem:
+ VERBOSE { $$ = VACOPT_VERBOSE; }
;
analyze_keyword:
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index c440c7e..199dda6 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -112,6 +112,9 @@ ANALYZE vactst, does_not_exist, vacparted;
ERROR: relation "does_not_exist" does not exist
ANALYZE vactst (i), vacparted (does_not_exist);
ERROR: column "does_not_exist" of relation "vacparted" does not exist
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 92eaca2..e7b7a7f 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -89,6 +89,9 @@ ANALYZE vacparted (b), vactst;
ANALYZE vactst, does_not_exist, vacparted;
ANALYZE vactst (i), vacparted (does_not_exist);
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
On 1/9/18, 8:55 PM, "Michael Paquier" <michael.paquier@gmail.com> wrote:
On Tue, Jan 09, 2018 at 01:46:55PM -0800, Andres Freund wrote:
On 2018-01-09 10:23:12 -0500, Robert Haas wrote:
On Mon, Jan 8, 2018 at 11:27 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Dec 28, 2017 at 10:46:18PM +0000, Bossart, Nathan wrote:
I agree, this makes more sense. I've made this change in v3 of 0003.
Based on the opinions gathered on this thread, 0001 and 0002 seem to be
in the shape wanted, and those look good for shipping.Committed 0001.
FWIW, I think in the future it'd be good to develop new features outside
of a thread about "vacuum crashes".You are right atht this is bad practice. Partially my fault for not
recommending the move to a new thread.
Yes, sorry about that. I should have opened a new thread.
Nathan
On Wed, Jan 10, 2018 at 05:26:43PM +0000, Bossart, Nathan wrote:
Right. I don't have a terribly strong opinion either way. I think the
counter-argument is that logging skipped relations might provide
valuable feedback to users. If I ran a database-wide VACUUM and a
relation was skipped due to lock contention, it might be nice to know
that so I can handle the relation individually.
Or users may not care about getting information they don't care
about. When running a database-wide VACUUM or ANALYZE you don't know
what is exactly the list of relations this is working on. Getting
information about things you don't know beforehand can be considered as
misinformation.
Perhaps others have different opinions. Sawada-san?
Perhaps any logging changes for VACOPT_NOWAIT could be handled in a
separate thread. For now, I've updated 0003 to remove the logging
changes.
Thanks. I am marking those as ready for committer, you are providing the
set patch patch which offer the most consistent experience.
--
Michael
On Thu, Jan 11, 2018 at 8:14 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Wed, Jan 10, 2018 at 05:26:43PM +0000, Bossart, Nathan wrote:
Right. I don't have a terribly strong opinion either way. I think the
counter-argument is that logging skipped relations might provide
valuable feedback to users. If I ran a database-wide VACUUM and a
relation was skipped due to lock contention, it might be nice to know
that so I can handle the relation individually.Or users may not care about getting information they don't care
about. When running a database-wide VACUUM or ANALYZE you don't know
what is exactly the list of relations this is working on. Getting
information about things you don't know beforehand can be considered as
misinformation.Perhaps others have different opinions. Sawada-san?
Hmm, I agree that v4 patch is more simple and consistent with current
behavior. The logging is not necessary if relation has been dropped
but I think that it's necessary if a relation exists but database-wide
VACUUM or ANALYZE could not take the lock on it. I can image a use
case where a user don't rely on autovacuum and do manual vacuums in a
maintenance window (per-week or per-day). In this case the log message
of skipping vacuum would be useful for user. The user might not be
interested in what is exactly the list of relations but might be
interested in the relations that were skipped to vacuum. IIUC since
autovacuum always passes a list of VacuumRelation (length is 1 and
having non-NULL RangeVar) to vacuum(), if autovacuum could not take
the lock on the relation it certainly emits that log. But with v4
patch, that log message is not emitted only if user do database-wide
VACUUM or ANALYZE and could not take the lock. On the other hand, the
downside of that we emits that log even if relations == NULL is we
emits that log even for toast tables and child tables. It would be
annoy user and might be unnecessary information. To deal with it, if
NOWAIT is specified we can fill RangeVar in get_all_vacuum_rels and
make vacuum_rel not emit "relation no longer exists" log. That way we
can emits that log by database-wide VACUUM/ANALYZE, while not emitting
for toast table and child tables. But I'm not sure it's worth to
effort for this direction (v3patch + tricks) going so far as making
such tricks. I'd like to hear opinions.
Regards,
--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On 2018-01-11 08:14:42 +0900, Michael Paquier wrote:
On Wed, Jan 10, 2018 at 05:26:43PM +0000, Bossart, Nathan wrote:
Perhaps any logging changes for VACOPT_NOWAIT could be handled in a
separate thread. For now, I've updated 0003 to remove the logging
changes.Thanks. I am marking those as ready for committer, you are providing the
set patch patch which offer the most consistent experience.
I was working on committing 0002 and 0003, when I noticed that the
second patch doesn't actually fully works. NOWAIT does what it says on
the tin iff the table is locked with a lower lock level than access
exclusive. But if AEL is used, the command is blocked in
static List *
expand_vacuum_rel(VacuumRelation *vrel)
...
/*
* We transiently take AccessShareLock to protect the syscache lookup
* below, as well as find_all_inheritors's expectation that the caller
* holds some lock on the starting relation.
*/
relid = RangeVarGetRelid(vrel->relation, AccessShareLock, false);
ISTM has been added after the patches initially were proposed. See
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=19de0ab23ccba12567c18640f00b49f01471018d
I'm a bit disappointed that the tests didn't catch this, they're clearly
not fully there. They definitely should test the AEL case, as
demonstrated here.
Independent of that, I'm also concerned that NOWAIT isn't implemented
consistently with other commands. Aren't we erroring out in other uses
of NOWAIT? ISTM a more appropriate keyword would have been SKIP
LOCKED. I think the behaviour makes sense, but I'd rename the internal
flag and the grammar to use SKIP LOCKED.
Lightly edited patches attached. Please preserve commit messages while
fixing these issues.
Greetings,
Andres Freund
Attachments:
v5-0001-Add-parenthesized-options-syntax-for-ANALYZE.patchtext/x-diff; charset=us-asciiDownload
From 3a59265b8ae9837a16aed76773d638f7392a395b Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Sat, 3 Mar 2018 15:47:53 -0800
Subject: [PATCH v5 1/2] Add parenthesized options syntax for ANALYZE.
This is analogous to the syntax allowed for VACUUM. This allows us to
avoid making new options reserved keywords and makes it easier to
allow arbitrary argument order. Oh, and it's consistent with the other
commands, too.
Author: Nathan Bossart
Reviewed-By: Michael Paquier, Masahiko Sawada
Discussion: https://postgr.es/m/D3FC73E2-9B1A-4DB4-8180-55F57D116B4E@amazon.com
---
doc/src/sgml/ref/analyze.sgml | 14 +++++++++++++-
src/backend/parser/gram.y | 17 +++++++++++++++++
src/test/regress/expected/vacuum.out | 7 +++++++
src/test/regress/sql/vacuum.sql | 4 ++++
4 files changed, 41 insertions(+), 1 deletion(-)
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 83b07a03003..10b3a9a6733 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -21,9 +21,14 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
+ANALYZE [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
-<phrase>where <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
+<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
+
+ VERBOSE
+
+<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
<replaceable class="parameter">table_name</replaceable> [ ( <replaceable class="parameter">column_name</replaceable> [, ...] ) ]
</synopsis>
@@ -49,6 +54,13 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
It is further possible to give a list of column names for a table,
in which case only the statistics for those columns are collected.
</para>
+
+ <para>
+ When the option list is surrounded by parentheses, the options can be
+ written in any order. The parenthesized syntax was added in
+ <productname>PostgreSQL</productname> 11; the unparenthesized syntax
+ is deprecated.
+ </para>
</refsect1>
<refsect1>
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d99f2be2c97..1d7317c6eac 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -306,6 +306,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> opt_lock lock_type cast_context
%type <ival> vacuum_option_list vacuum_option_elem
+ analyze_option_list analyze_option_elem
%type <boolean> opt_or_replace
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
@@ -10548,6 +10549,22 @@ AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
n->rels = $3;
$$ = (Node *)n;
}
+ | analyze_keyword '(' analyze_option_list ')' opt_vacuum_relation_list
+ {
+ VacuumStmt *n = makeNode(VacuumStmt);
+ n->options = VACOPT_ANALYZE | $3;
+ n->rels = $5;
+ $$ = (Node *) n;
+ }
+ ;
+
+analyze_option_list:
+ analyze_option_elem { $$ = $1; }
+ | analyze_option_list ',' analyze_option_elem { $$ = $1 | $3; }
+ ;
+
+analyze_option_elem:
+ VERBOSE { $$ = VACOPT_VERBOSE; }
;
analyze_keyword:
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index c440c7ea58f..d66e2aa3b70 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -112,6 +112,13 @@ ANALYZE vactst, does_not_exist, vacparted;
ERROR: relation "does_not_exist" does not exist
ANALYZE vactst (i), vacparted (does_not_exist);
ERROR: column "does_not_exist" of relation "vacparted" does not exist
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
+ANALYZE (nonexistant-arg) does_not_exist;
+ERROR: syntax error at or near "nonexistant"
+LINE 1: ANALYZE (nonexistant-arg) does_not_exist;
+ ^
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 92eaca2a93b..275ce2e270f 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -89,6 +89,10 @@ ANALYZE vacparted (b), vactst;
ANALYZE vactst, does_not_exist, vacparted;
ANALYZE vactst (i), vacparted (does_not_exist);
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+ANALYZE (nonexistant-arg) does_not_exist;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
--
2.15.1.354.g95ec6b1b33.dirty
v5-0002-Add-NOWAIT-option-to-VACUUM-and-ANALYZE.patchtext/x-diff; charset=us-asciiDownload
From 1f38a54097328bb80a279d63474fdf31bf52fabe Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Sat, 3 Mar 2018 16:09:18 -0800
Subject: [PATCH v5 2/2] Add NOWAIT option to VACUUM and ANALYZE.
FIXME: blocking doesn't work for AEL
FIXME: NOWAIT isn't actually implemented, closer to SKIP LOCKED
Author: Nathan Bossart
Reviewed-By: Michael Paquier, Andres Freund, Masahiko Sawada
Discussion: https://postgr.es/m/D3FC73E2-9B1A-4DB4-8180-55F57D116B4E@amazon.com
---
doc/src/sgml/ref/analyze.sgml | 12 +++++
doc/src/sgml/ref/vacuum.sgml | 12 +++++
src/backend/parser/gram.y | 2 +
src/include/nodes/parsenodes.h | 2 +-
src/test/isolation/expected/vacuum-nowait.out | 64 +++++++++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
src/test/isolation/specs/vacuum-nowait.spec | 42 ++++++++++++++++++
src/test/regress/expected/vacuum.out | 9 ++++
src/test/regress/sql/vacuum.sql | 8 ++++
9 files changed, 151 insertions(+), 1 deletion(-)
create mode 100644 src/test/isolation/expected/vacuum-nowait.out
create mode 100644 src/test/isolation/specs/vacuum-nowait.spec
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 10b3a9a6733..05882c30328 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -27,6 +27,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
VERBOSE
+ NOWAIT
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -76,6 +77,17 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>NOWAIT</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>ANALYZE</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index b760e8ede18..1d7ffef734f 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -31,6 +31,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
VERBOSE
ANALYZE
DISABLE_PAGE_SKIPPING
+ NOWAIT
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -160,6 +161,17 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>NOWAIT</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 1d7317c6eac..54ac7aaaee6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -10538,6 +10538,7 @@ vacuum_option_elem:
errmsg("unrecognized VACUUM option \"%s\"", $1),
parser_errposition(@1)));
}
+ | NOWAIT { $$ = VACOPT_NOWAIT; }
;
AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
@@ -10565,6 +10566,7 @@ analyze_option_list:
analyze_option_elem:
VERBOSE { $$ = VACOPT_VERBOSE; }
+ | NOWAIT { $$ = VACOPT_NOWAIT; }
;
analyze_keyword:
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ac292bc6e7a..56e22ae29a2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3125,7 +3125,7 @@ typedef enum VacuumOption
VACOPT_VERBOSE = 1 << 2, /* print progress info */
VACOPT_FREEZE = 1 << 3, /* FREEZE option */
VACOPT_FULL = 1 << 4, /* FULL (non-concurrent) vacuum */
- VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock (autovacuum only) */
+ VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock */
VACOPT_SKIPTOAST = 1 << 6, /* don't process the TOAST table, if any */
VACOPT_DISABLE_PAGE_SKIPPING = 1 << 7 /* don't skip any pages */
} VacuumOption;
diff --git a/src/test/isolation/expected/vacuum-nowait.out b/src/test/isolation/expected/vacuum-nowait.out
new file mode 100644
index 00000000000..225ec23760e
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-nowait.out
@@ -0,0 +1,64 @@
+Parsed test spec with 2 sessions
+
+starting permutation: lock vac_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_specified: VACUUM (NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step vac_all_parts: VACUUM (NOWAIT) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock analyze_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping analyze of "part1" --- lock not available
+step analyze_specified: ANALYZE (NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock analyze_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step analyze_all_parts: ANALYZE (NOWAIT) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_analyze_specified commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_analyze_specified: VACUUM (ANALYZE, NOWAIT) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock vac_analyze_all_parts commit
+step lock:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step vac_analyze_all_parts: VACUUM (ANALYZE, NOWAIT) parted;
+step commit:
+ COMMIT;
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 74d7d59546a..140b637eafa 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -66,3 +66,4 @@ test: async-notify
test: vacuum-reltuples
test: timeouts
test: vacuum-concurrent-drop
+test: vacuum-nowait
diff --git a/src/test/isolation/specs/vacuum-nowait.spec b/src/test/isolation/specs/vacuum-nowait.spec
new file mode 100644
index 00000000000..498b0a0583a
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-nowait.spec
@@ -0,0 +1,42 @@
+# Test for the NOWAIT option for VACUUM and ANALYZE commands.
+#
+# This also verifies that log messages are not emitted for skipped relations
+# that were not specified in the VACUUM or ANALYZE command.
+
+setup
+{
+ CREATE TABLE parted (a INT) PARTITION BY LIST (a);
+ CREATE TABLE part1 PARTITION OF parted FOR VALUES IN (1);
+ CREATE TABLE part2 PARTITION OF parted FOR VALUES IN (2);
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS parted;
+}
+
+session "s1"
+step "lock"
+{
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+}
+step "commit"
+{
+ COMMIT;
+}
+
+session "s2"
+step "vac_specified" { VACUUM (NOWAIT) part1, part2; }
+step "vac_all_parts" { VACUUM (NOWAIT) parted; }
+step "analyze_specified" { ANALYZE (NOWAIT) part1, part2; }
+step "analyze_all_parts" { ANALYZE (NOWAIT) parted; }
+step "vac_analyze_specified" { VACUUM (ANALYZE, NOWAIT) part1, part2; }
+step "vac_analyze_all_parts" { VACUUM (ANALYZE, NOWAIT) parted; }
+
+permutation "lock" "vac_specified" "commit"
+permutation "lock" "vac_all_parts" "commit"
+permutation "lock" "analyze_specified" "commit"
+permutation "lock" "analyze_all_parts" "commit"
+permutation "lock" "vac_analyze_specified" "commit"
+permutation "lock" "vac_analyze_all_parts" "commit"
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index d66e2aa3b70..f06d975f26c 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -119,6 +119,15 @@ ANALYZE (nonexistant-arg) does_not_exist;
ERROR: syntax error at or near "nonexistant"
LINE 1: ANALYZE (nonexistant-arg) does_not_exist;
^
+-- ensure argument order independence, and that NOWAIT on non-existing
+-- relation still errors out.
+ANALYZE (NOWAIT, VERBOSE) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
+ANALYZE (VERBOSE, NOWAIT) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
+-- nowait option
+VACUUM (NOWAIT) vactst;
+ANALYZE (NOWAIT) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 275ce2e270f..5fb6ad20051 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -92,6 +92,14 @@ ANALYZE vactst (i), vacparted (does_not_exist);
-- parenthesized syntax for ANALYZE
ANALYZE (VERBOSE) does_not_exist;
ANALYZE (nonexistant-arg) does_not_exist;
+-- ensure argument order independence, and that NOWAIT on non-existing
+-- relation still errors out.
+ANALYZE (NOWAIT, VERBOSE) does_not_exist;
+ANALYZE (VERBOSE, NOWAIT) does_not_exist;
+
+-- nowait option
+VACUUM (NOWAIT) vactst;
+ANALYZE (NOWAIT) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
--
2.15.1.354.g95ec6b1b33.dirty
Thanks for taking a look.
On 3/3/18, 6:13 PM, "Andres Freund" <andres@anarazel.de> wrote:
I was working on committing 0002 and 0003, when I noticed that the
second patch doesn't actually fully works. NOWAIT does what it says on
the tin iff the table is locked with a lower lock level than access
exclusive. But if AEL is used, the command is blocked instatic List *
expand_vacuum_rel(VacuumRelation *vrel)
...
/*
* We transiently take AccessShareLock to protect the syscache lookup
* below, as well as find_all_inheritors's expectation that the caller
* holds some lock on the starting relation.
*/
relid = RangeVarGetRelid(vrel->relation, AccessShareLock, false);ISTM has been added after the patches initially were proposed. See
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=19de0ab23ccba12567c18640f00b49f01471018dI'm a bit disappointed that the tests didn't catch this, they're clearly
not fully there. They definitely should test the AEL case, as
demonstrated here.
Sorry about that. I've added logic to handle ACCESS EXCLUSIVE in v6 of 0002,
and I've extended the tests to cover that case.
Independent of that, I'm also concerned that NOWAIT isn't implemented
consistently with other commands. Aren't we erroring out in other uses
of NOWAIT? ISTM a more appropriate keyword would have been SKIP
LOCKED. I think the behaviour makes sense, but I'd rename the internal
flag and the grammar to use SKIP LOCKED.
Agreed. I've updated 0002 to use SKIP LOCKED instead of NOWAIT.
Nathan
Attachments:
v5-0001-Add-parenthesized-options-syntax-for-ANALYZE.patchapplication/octet-stream; name=v5-0001-Add-parenthesized-options-syntax-for-ANALYZE.patchDownload
From 3a59265b8ae9837a16aed76773d638f7392a395b Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Sat, 3 Mar 2018 15:47:53 -0800
Subject: [PATCH v5 1/2] Add parenthesized options syntax for ANALYZE.
This is analogous to the syntax allowed for VACUUM. This allows us to
avoid making new options reserved keywords and makes it easier to
allow arbitrary argument order. Oh, and it's consistent with the other
commands, too.
Author: Nathan Bossart
Reviewed-By: Michael Paquier, Masahiko Sawada
Discussion: https://postgr.es/m/D3FC73E2-9B1A-4DB4-8180-55F57D116B4E@amazon.com
---
doc/src/sgml/ref/analyze.sgml | 14 +++++++++++++-
src/backend/parser/gram.y | 17 +++++++++++++++++
src/test/regress/expected/vacuum.out | 7 +++++++
src/test/regress/sql/vacuum.sql | 4 ++++
4 files changed, 41 insertions(+), 1 deletion(-)
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 83b07a03003..10b3a9a6733 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -21,9 +21,14 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
+ANALYZE [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replaceable> [, ...] ]
-<phrase>where <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
+<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
+
+ VERBOSE
+
+<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
<replaceable class="parameter">table_name</replaceable> [ ( <replaceable class="parameter">column_name</replaceable> [, ...] ) ]
</synopsis>
@@ -49,6 +54,13 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
It is further possible to give a list of column names for a table,
in which case only the statistics for those columns are collected.
</para>
+
+ <para>
+ When the option list is surrounded by parentheses, the options can be
+ written in any order. The parenthesized syntax was added in
+ <productname>PostgreSQL</productname> 11; the unparenthesized syntax
+ is deprecated.
+ </para>
</refsect1>
<refsect1>
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d99f2be2c97..1d7317c6eac 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -306,6 +306,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> opt_lock lock_type cast_context
%type <ival> vacuum_option_list vacuum_option_elem
+ analyze_option_list analyze_option_elem
%type <boolean> opt_or_replace
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
@@ -10548,6 +10549,22 @@ AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
n->rels = $3;
$$ = (Node *)n;
}
+ | analyze_keyword '(' analyze_option_list ')' opt_vacuum_relation_list
+ {
+ VacuumStmt *n = makeNode(VacuumStmt);
+ n->options = VACOPT_ANALYZE | $3;
+ n->rels = $5;
+ $$ = (Node *) n;
+ }
+ ;
+
+analyze_option_list:
+ analyze_option_elem { $$ = $1; }
+ | analyze_option_list ',' analyze_option_elem { $$ = $1 | $3; }
+ ;
+
+analyze_option_elem:
+ VERBOSE { $$ = VACOPT_VERBOSE; }
;
analyze_keyword:
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index c440c7ea58f..d66e2aa3b70 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -112,6 +112,13 @@ ANALYZE vactst, does_not_exist, vacparted;
ERROR: relation "does_not_exist" does not exist
ANALYZE vactst (i), vacparted (does_not_exist);
ERROR: column "does_not_exist" of relation "vacparted" does not exist
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
+ANALYZE (nonexistant-arg) does_not_exist;
+ERROR: syntax error at or near "nonexistant"
+LINE 1: ANALYZE (nonexistant-arg) does_not_exist;
+ ^
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 92eaca2a93b..275ce2e270f 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -89,6 +89,10 @@ ANALYZE vacparted (b), vactst;
ANALYZE vactst, does_not_exist, vacparted;
ANALYZE vactst (i), vacparted (does_not_exist);
+-- parenthesized syntax for ANALYZE
+ANALYZE (VERBOSE) does_not_exist;
+ANALYZE (nonexistant-arg) does_not_exist;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
--
2.15.1.354.g95ec6b1b33.dirty
v6-0002-Add-NOWAIT-option-to-VACUUM-and-ANALYZE.patchapplication/octet-stream; name=v6-0002-Add-NOWAIT-option-to-VACUUM-and-ANALYZE.patchDownload
From 05299c00c6933eaf0e97b8b5f512e190c5abe8d3 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Mon, 5 Mar 2018 21:46:01 +0000
Subject: [PATCH v6 2/2] Add NOWAIT option to VACUUM and ANALYZE.
FIXME: blocking doesn't work for AEL
FIXME: NOWAIT isn't actually implemented, closer to SKIP LOCKED
Author: Nathan Bossart
Reviewed-By: Michael Paquier, Andres Freund, Masahiko Sawada
Discussion: https://postgr.es/m/D3FC73E2-9B1A-4DB4-8180-55F57D116B4E@amazon.com
---
doc/src/sgml/ref/analyze.sgml | 12 ++
doc/src/sgml/ref/vacuum.sgml | 12 ++
src/backend/commands/analyze.c | 2 +-
src/backend/commands/vacuum.c | 55 ++++++++-
src/backend/parser/gram.y | 2 +
src/backend/postmaster/autovacuum.c | 2 +-
src/include/nodes/parsenodes.h | 2 +-
src/test/isolation/expected/vacuum-skip-locked.out | 129 +++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
src/test/isolation/specs/vacuum-skip-locked.spec | 53 +++++++++
src/test/regress/expected/vacuum.out | 9 ++
src/test/regress/sql/vacuum.sql | 9 ++
12 files changed, 280 insertions(+), 8 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-skip-locked.out
create mode 100644 src/test/isolation/specs/vacuum-skip-locked.spec
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 10b3a9a..4e6ae38 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -27,6 +27,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
VERBOSE
+ SKIP LOCKED
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -77,6 +78,17 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
</varlistentry>
<varlistentry>
+ <term><literal>SKIP LOCKED</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>ANALYZE</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index b760e8e..1ef5bc9 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -31,6 +31,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
VERBOSE
ANALYZE
DISABLE_PAGE_SKIPPING
+ SKIP LOCKED
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -161,6 +162,17 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</varlistentry>
<varlistentry>
+ <term><literal>SKIP LOCKED</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
<para>
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 5f21fcb..68f84c9 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -143,7 +143,7 @@ analyze_rel(Oid relid, RangeVar *relation, int options,
* matter if we ever try to accumulate stats on dead tuples.) If the rel
* has been dropped since we last saw it, we don't need to process it.
*/
- if (!(options & VACOPT_NOWAIT))
+ if (!(options & VACOPT_SKIP_LOCKED))
onerel = try_relation_open(relid, ShareUpdateExclusiveLock);
else if (ConditionalLockRelationOid(relid, ShareUpdateExclusiveLock))
onerel = try_relation_open(relid, NoLock);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 7aca69a..62eb95c 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -46,6 +46,7 @@
#include "storage/procarray.h"
#include "utils/acl.h"
#include "utils/fmgroids.h"
+#include "utils/formatting.h"
#include "utils/guc.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -68,7 +69,7 @@ static BufferAccessStrategy vac_strategy;
/* non-export function prototypes */
-static List *expand_vacuum_rel(VacuumRelation *vrel);
+static List *expand_vacuum_rel(VacuumRelation *vrel, int options, const char *stmttype);
static List *get_all_vacuum_rels(void);
static void vac_truncate_clog(TransactionId frozenXID,
MultiXactId minMulti,
@@ -250,6 +251,13 @@ vacuum(int options, List *relations, VacuumParams *params,
{
List *newrels = NIL;
ListCell *lc;
+ const char *stmttype_lower;
+
+ /*
+ * We must downcase the statement type for log message consistency
+ * between expand_vacuum_rel(), vacuum_rel(), and analyze_rel().
+ */
+ stmttype_lower = asc_tolower(stmttype, strlen(stmttype));
foreach(lc, relations)
{
@@ -257,7 +265,7 @@ vacuum(int options, List *relations, VacuumParams *params,
List *sublist;
MemoryContext old_context;
- sublist = expand_vacuum_rel(vrel);
+ sublist = expand_vacuum_rel(vrel, options, stmttype_lower);
old_context = MemoryContextSwitchTo(vac_context);
newrels = list_concat(newrels, sublist);
MemoryContextSwitchTo(old_context);
@@ -423,7 +431,7 @@ vacuum(int options, List *relations, VacuumParams *params,
* are made in vac_context.
*/
static List *
-expand_vacuum_rel(VacuumRelation *vrel)
+expand_vacuum_rel(VacuumRelation *vrel, int options, const char *stmttype)
{
List *vacrels = NIL;
MemoryContext oldcontext;
@@ -444,11 +452,48 @@ expand_vacuum_rel(VacuumRelation *vrel)
bool include_parts;
/*
+ * Since autovacuum workers supply OIDs when calling vacuum(), no
+ * autovacuum worker should reach this code, and we can make
+ * assumptions about the logging levels we should use in the checks
+ * below.
+ */
+ Assert(!IsAutoVacuumWorkerProcess());
+
+ /*
* We transiently take AccessShareLock to protect the syscache lookup
* below, as well as find_all_inheritors's expectation that the caller
* holds some lock on the starting relation.
*/
- relid = RangeVarGetRelid(vrel->relation, AccessShareLock, false);
+ if (!(options & VACOPT_SKIP_LOCKED))
+ relid = RangeVarGetRelid(vrel->relation, AccessShareLock, false);
+ else
+ {
+ relid = RangeVarGetRelid(vrel->relation, NoLock, false);
+
+ /*
+ * The relation may go away before we can obtain the lock, so we
+ * must verify that it still exists afterwards. If the lock is
+ * unavailable or the relation has disappeared, emit the same log
+ * statement that vacuum_rel() and analyze_rel() would.
+ */
+ if (!ConditionalLockRelationOid(relid, AccessShareLock))
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
+ errmsg("skipping %s of \"%s\" --- lock not available",
+ stmttype, vrel->relation->relname)));
+ return vacrels;
+ }
+ else if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relid)))
+ {
+ UnlockRelationOid(relid, AccessShareLock);
+ ereport(WARNING,
+ (errcode(ERRCODE_UNDEFINED_TABLE),
+ errmsg("skipping %s of \"%s\" --- relation no longer exists",
+ stmttype, vrel->relation->relname)));
+ return vacrels;
+ }
+ }
/*
* Make a returnable VacuumRelation for this rel.
@@ -1395,7 +1440,7 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
* If we've been asked not to wait for the relation lock, acquire it first
* in non-blocking mode, before calling try_relation_open().
*/
- if (!(options & VACOPT_NOWAIT))
+ if (!(options & VACOPT_SKIP_LOCKED))
onerel = try_relation_open(relid, lmode);
else if (ConditionalLockRelationOid(relid, lmode))
onerel = try_relation_open(relid, NoLock);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 1d7317c..977a030 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -10538,6 +10538,7 @@ vacuum_option_elem:
errmsg("unrecognized VACUUM option \"%s\"", $1),
parser_errposition(@1)));
}
+ | SKIP LOCKED { $$ = VACOPT_SKIP_LOCKED; }
;
AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
@@ -10565,6 +10566,7 @@ analyze_option_list:
analyze_option_elem:
VERBOSE { $$ = VACOPT_VERBOSE; }
+ | SKIP LOCKED { $$ = VACOPT_SKIP_LOCKED; }
;
analyze_keyword:
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 21f5e2c..77cbfe9 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -2883,7 +2883,7 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
tab->at_vacoptions = VACOPT_SKIPTOAST |
(dovacuum ? VACOPT_VACUUM : 0) |
(doanalyze ? VACOPT_ANALYZE : 0) |
- (!wraparound ? VACOPT_NOWAIT : 0);
+ (!wraparound ? VACOPT_SKIP_LOCKED : 0);
tab->at_params.freeze_min_age = freeze_min_age;
tab->at_params.freeze_table_age = freeze_table_age;
tab->at_params.multixact_freeze_min_age = multixact_freeze_min_age;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ac292bc..744fdc4 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3125,7 +3125,7 @@ typedef enum VacuumOption
VACOPT_VERBOSE = 1 << 2, /* print progress info */
VACOPT_FREEZE = 1 << 3, /* FREEZE option */
VACOPT_FULL = 1 << 4, /* FULL (non-concurrent) vacuum */
- VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock (autovacuum only) */
+ VACOPT_SKIP_LOCKED = 1 << 5, /* don't wait to get lock */
VACOPT_SKIPTOAST = 1 << 6, /* don't process the TOAST table, if any */
VACOPT_DISABLE_PAGE_SKIPPING = 1 << 7 /* don't skip any pages */
} VacuumOption;
diff --git a/src/test/isolation/expected/vacuum-skip-locked.out b/src/test/isolation/expected/vacuum-skip-locked.out
new file mode 100644
index 0000000..cb11672
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-skip-locked.out
@@ -0,0 +1,129 @@
+Parsed test spec with 2 sessions
+
+starting permutation: lock_share vac_specified commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_specified: VACUUM (SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share vac_all_parts commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step vac_all_parts: VACUUM (SKIP LOCKED) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share analyze_specified commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping analyze of "part1" --- lock not available
+step analyze_specified: ANALYZE (SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share analyze_all_parts commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step analyze_all_parts: ANALYZE (SKIP LOCKED) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share vac_analyze_specified commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_analyze_specified: VACUUM (ANALYZE, SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share vac_analyze_all_parts commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step vac_analyze_all_parts: VACUUM (ANALYZE, SKIP LOCKED) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive vac_specified commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_specified: VACUUM (SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive vac_all_parts commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+step vac_all_parts: VACUUM (SKIP LOCKED) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive analyze_specified commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+WARNING: skipping analyze of "part1" --- lock not available
+step analyze_specified: ANALYZE (SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive analyze_all_parts commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+step analyze_all_parts: ANALYZE (SKIP LOCKED) parted; <waiting ...>
+step commit:
+ COMMIT;
+
+step analyze_all_parts: <... completed>
+
+starting permutation: lock_access_exclusive vac_analyze_specified commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_analyze_specified: VACUUM (ANALYZE, SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive vac_analyze_all_parts commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+step vac_analyze_all_parts: VACUUM (ANALYZE, SKIP LOCKED) parted; <waiting ...>
+step commit:
+ COMMIT;
+
+step vac_analyze_all_parts: <... completed>
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 74d7d59..520734b 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -66,3 +66,4 @@ test: async-notify
test: vacuum-reltuples
test: timeouts
test: vacuum-concurrent-drop
+test: vacuum-skip-locked
diff --git a/src/test/isolation/specs/vacuum-skip-locked.spec b/src/test/isolation/specs/vacuum-skip-locked.spec
new file mode 100644
index 0000000..96c3bf4
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-skip-locked.spec
@@ -0,0 +1,53 @@
+# Test for the SKIP LOCKED option for VACUUM and ANALYZE commands.
+#
+# This also verifies that log messages are not emitted for skipped relations
+# that were not specified in the VACUUM or ANALYZE command.
+
+setup
+{
+ CREATE TABLE parted (a INT) PARTITION BY LIST (a);
+ CREATE TABLE part1 PARTITION OF parted FOR VALUES IN (1);
+ CREATE TABLE part2 PARTITION OF parted FOR VALUES IN (2);
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS parted;
+}
+
+session "s1"
+step "lock_share"
+{
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+}
+step "lock_access_exclusive"
+{
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+}
+step "commit"
+{
+ COMMIT;
+}
+
+session "s2"
+step "vac_specified" { VACUUM (SKIP LOCKED) part1, part2; }
+step "vac_all_parts" { VACUUM (SKIP LOCKED) parted; }
+step "analyze_specified" { ANALYZE (SKIP LOCKED) part1, part2; }
+step "analyze_all_parts" { ANALYZE (SKIP LOCKED) parted; }
+step "vac_analyze_specified" { VACUUM (ANALYZE, SKIP LOCKED) part1, part2; }
+step "vac_analyze_all_parts" { VACUUM (ANALYZE, SKIP LOCKED) parted; }
+
+permutation "lock_share" "vac_specified" "commit"
+permutation "lock_share" "vac_all_parts" "commit"
+permutation "lock_share" "analyze_specified" "commit"
+permutation "lock_share" "analyze_all_parts" "commit"
+permutation "lock_share" "vac_analyze_specified" "commit"
+permutation "lock_share" "vac_analyze_all_parts" "commit"
+permutation "lock_access_exclusive" "vac_specified" "commit"
+permutation "lock_access_exclusive" "vac_all_parts" "commit"
+permutation "lock_access_exclusive" "analyze_specified" "commit"
+permutation "lock_access_exclusive" "analyze_all_parts" "commit"
+permutation "lock_access_exclusive" "vac_analyze_specified" "commit"
+permutation "lock_access_exclusive" "vac_analyze_all_parts" "commit"
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index d66e2aa..5adfdb1 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -119,6 +119,15 @@ ANALYZE (nonexistant-arg) does_not_exist;
ERROR: syntax error at or near "nonexistant"
LINE 1: ANALYZE (nonexistant-arg) does_not_exist;
^
+-- ensure argument order independence, and that SKIP LOCKED on non-existing
+-- relation still errors out.
+ANALYZE (SKIP LOCKED, VERBOSE) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
+ANALYZE (VERBOSE, SKIP LOCKED) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
+-- nowait option
+VACUUM (SKIP LOCKED) vactst;
+ANALYZE (SKIP LOCKED) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 275ce2e..908a288 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -93,6 +93,15 @@ ANALYZE vactst (i), vacparted (does_not_exist);
ANALYZE (VERBOSE) does_not_exist;
ANALYZE (nonexistant-arg) does_not_exist;
+-- ensure argument order independence, and that SKIP LOCKED on non-existing
+-- relation still errors out.
+ANALYZE (SKIP LOCKED, VERBOSE) does_not_exist;
+ANALYZE (VERBOSE, SKIP LOCKED) does_not_exist;
+
+-- nowait option
+VACUUM (SKIP LOCKED) vactst;
+ANALYZE (SKIP LOCKED) vactst;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
--
2.7.3.AMZN
On 2018-03-03 16:12:52 -0800, Andres Freund wrote:
On 2018-01-11 08:14:42 +0900, Michael Paquier wrote:
On Wed, Jan 10, 2018 at 05:26:43PM +0000, Bossart, Nathan wrote:
Perhaps any logging changes for VACOPT_NOWAIT could be handled in a
separate thread. For now, I've updated 0003 to remove the logging
changes.Thanks. I am marking those as ready for committer, you are providing the
set patch patch which offer the most consistent experience.I was working on committing 0002 and 0003, when I noticed that the
second patch doesn't actually fully works. NOWAIT does what it says on
the tin iff the table is locked with a lower lock level than access
exclusive. But if AEL is used, the command is blocked in
I've pushed the first one now. There seems to be no reason to wait with
it, even it takes a bit till we can get the second part right.
- Andres
On 3/5/18, 7:58 PM, "Andres Freund" <andres@anarazel.de> wrote:
I've pushed the first one now. There seems to be no reason to wait with
it, even it takes a bit till we can get the second part right.
Thanks!
Nathan
On Mon, Mar 05, 2018 at 09:55:13PM +0000, Bossart, Nathan wrote:
On 3/3/18, 6:13 PM, "Andres Freund" <andres@anarazel.de> wrote:
I was working on committing 0002 and 0003, when I noticed that the
second patch doesn't actually fully works. NOWAIT does what it says on
the tin iff the table is locked with a lower lock level than access
exclusive. But if AEL is used, the command is blocked instatic List *
expand_vacuum_rel(VacuumRelation *vrel)
...
/*
* We transiently take AccessShareLock to protect the syscache lookup
* below, as well as find_all_inheritors's expectation that the caller
* holds some lock on the starting relation.
*/
relid = RangeVarGetRelid(vrel->relation, AccessShareLock, false);ISTM has been added after the patches initially were proposed. See
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=19de0ab23ccba12567c18640f00b49f01471018dI'm a bit disappointed that the tests didn't catch this, they're clearly
not fully there. They definitely should test the AEL case, as
demonstrated here.Sorry about that. I've added logic to handle ACCESS EXCLUSIVE in v6 of 0002,
and I've extended the tests to cover that case.
Hm, yes. I have also let those patches rot a bit myself. Sorry for
that.
Independent of that, I'm also concerned that NOWAIT isn't implemented
consistently with other commands. Aren't we erroring out in other uses
of NOWAIT? ISTM a more appropriate keyword would have been SKIP
LOCKED. I think the behaviour makes sense, but I'd rename the internal
flag and the grammar to use SKIP LOCKED.Agreed. I've updated 0002 to use SKIP LOCKED instead of NOWAIT.
So, NOWAIT means "please try to take a lock, don't wait for it and issue
an error immediately if the lock cannot be taken". SKIP_LOCKED means
"please try to take a lock, don't wait for it, but do not issue an error
if the lock cannot be taken and move on to next steps". I agree that
this is more consistent.
+ if (!(options & VACOPT_SKIP_LOCKED))
+ relid = RangeVarGetRelid(vrel->relation, AccessShareLock,
false);
+ else
+ {
+ relid = RangeVarGetRelid(vrel->relation, NoLock, false);
Yeah, I agree with Andres that getting all this logic done in
RangeVarGetRelidExtended would be cleaner. Let's see where the other
thread leads us to:
https://www.postgresql.org/message-id/20180306005349.b65whmvj7z6hbe2y%40alap3.anarazel.de
+ /*
+ * We must downcase the statement type for log message
consistency
+ * between expand_vacuum_rel(), vacuum_rel(), and analyze_rel().
+ */
+ stmttype_lower = asc_tolower(stmttype, strlen(stmttype));
This blows up on multi-byte characters and may report incorrect relation
name if double quotes are used, no?
+ /*
+ * Since autovacuum workers supply OIDs when calling vacuum(), no
+ * autovacuum worker should reach this code, and we can make
+ * assumptions about the logging levels we should use in the
checks
+ * below.
+ */
+ Assert(!IsAutoVacuumWorkerProcess());
This is a good idea, should be a separate patch as other folks tend to
be confused with relid handling in expand_vacuum_rel().
+ Specifies that <command>VACUUM</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped
Should this mention as well that no errors are raised, allowing the
process to move on with the next relations?
--
Michael
On 3/6/18, 11:04 PM, "Michael Paquier" <michael@paquier.xyz> wrote:
+ if (!(options & VACOPT_SKIP_LOCKED)) + relid = RangeVarGetRelid(vrel->relation, AccessShareLock, false); + else + { + relid = RangeVarGetRelid(vrel->relation, NoLock, false); Yeah, I agree with Andres that getting all this logic done in RangeVarGetRelidExtended would be cleaner. Let's see where the other thread leads us to: https://www.postgresql.org/message-id/20180306005349.b65whmvj7z6hbe2y%40alap3.anarazel.de
I had missed that thread. Thanks for pointing it out.
+ /* + * We must downcase the statement type for log message consistency + * between expand_vacuum_rel(), vacuum_rel(), and analyze_rel(). + */ + stmttype_lower = asc_tolower(stmttype, strlen(stmttype)); This blows up on multi-byte characters and may report incorrect relation name if double quotes are used, no?
At the moment, stmttype is either "VACUUM" or "ANALYZE", but I suppose
there still might be multi-byte risk in translations.
+ /* + * Since autovacuum workers supply OIDs when calling vacuum(), no + * autovacuum worker should reach this code, and we can make + * assumptions about the logging levels we should use in the checks + * below. + */ + Assert(!IsAutoVacuumWorkerProcess()); This is a good idea, should be a separate patch as other folks tend to be confused with relid handling in expand_vacuum_rel().
Will do.
+ Specifies that <command>VACUUM</command> should not wait for any + conflicting locks to be released: if a relation cannot be locked + immediately without waiting, the relation is skipped Should this mention as well that no errors are raised, allowing the process to move on with the next relations?
IMO that is already covered by saying the relation is skipped,
although I'm not against explicitly stating it, too. Perhaps we could
outline the logging behavior as well.
Nathan
On 3/7/18, 9:34 AM, "Bossart, Nathan" <bossartn@amazon.com> wrote:
On 3/6/18, 11:04 PM, "Michael Paquier" <michael@paquier.xyz> wrote:
+ if (!(options & VACOPT_SKIP_LOCKED)) + relid = RangeVarGetRelid(vrel->relation, AccessShareLock, false); + else + { + relid = RangeVarGetRelid(vrel->relation, NoLock, false); Yeah, I agree with Andres that getting all this logic done in RangeVarGetRelidExtended would be cleaner. Let's see where the other thread leads us to: https://www.postgresql.org/message-id/20180306005349.b65whmvj7z6hbe2y%40alap3.anarazel.deI had missed that thread. Thanks for pointing it out.
I've noticed one more problem with ACCESS EXCLUSIVE. If an ACCESS
EXCLUSIVE lock is held on a child relation of a partitioned table,
an ANALYZE on the partitioned table will be blocked at
acquire_inherited_sample_rows().
static int
acquire_inherited_sample_rows(Relation onerel, int elevel,
HeapTuple *rows, int targrows,
double *totalrows, double *totaldeadrows)
{
...
/*
* Find all members of inheritance set. We only need AccessShareLock on
* the children.
*/
tableOIDs =
find_all_inheritors(RelationGetRelid(onerel), AccessShareLock, NULL);
It also seems possible for the call to vac_open_indexes() in
do_analyze_rel() to block.
/*
* Open all indexes of the relation, and see if there are any analyzable
* columns in the indexes. We do not analyze index columns if there was
* an explicit column list in the ANALYZE command, however. If we are
* doing a recursive scan, we don't want to touch the parent's indexes at
* all.
*/
if (!inh)
vac_open_indexes(onerel, AccessShareLock, &nindexes, &Irel);
else
{
Irel = NULL;
nindexes = 0;
}
I think the most straightforward approach for fixing this is to add
skip-locked functionality in find_all_inheritors(),
find_inheritance_children(), and vac_open_indexes(), but I am curious
what others think.
Nathan
Hi,
On 2018-03-30 18:39:01 +0000, Bossart, Nathan wrote:
I've noticed one more problem with ACCESS EXCLUSIVE. If an ACCESS
EXCLUSIVE lock is held on a child relation of a partitioned table,
an ANALYZE on the partitioned table will be blocked at
acquire_inherited_sample_rows().static int
acquire_inherited_sample_rows(Relation onerel, int elevel,
HeapTuple *rows, int targrows,
double *totalrows, double *totaldeadrows)
{
...
/*
* Find all members of inheritance set. We only need AccessShareLock on
* the children.
*/
tableOIDs =
find_all_inheritors(RelationGetRelid(onerel), AccessShareLock, NULL);
Right.
It also seems possible for the call to vac_open_indexes() in
do_analyze_rel() to block.
Yup.
I think the most straightforward approach for fixing this is to add
skip-locked functionality in find_all_inheritors(),
find_inheritance_children(), and vac_open_indexes(), but I am curious
what others think.
I'm actually wondering if we shouldn't just ignore this problem. While
the other cases VACUUM (SKIP LOCKED) are intended to solve seem common,
these seem less so. But it'd be a bit weird, too..
Could you post a rebased version of the patch, with an incremental
addition of what you propose in a separate patch?
Greetings,
Andres Freund
On Fri, Mar 30, 2018 at 05:19:46PM -0700, Andres Freund wrote:
On 2018-03-30 18:39:01 +0000, Bossart, Nathan wrote:
I think the most straightforward approach for fixing this is to add
skip-locked functionality in find_all_inheritors(),
find_inheritance_children(), and vac_open_indexes(), but I am curious
what others think.
For vac_open_indexes() it may make the most sense to introduce a new
try_open_index which wraps on top of try_relation_open with checks on
the opened relation's relkind.
I'm actually wondering if we shouldn't just ignore this problem. While
the other cases VACUUM (SKIP LOCKED) are intended to solve seem common,
these seem less so. But it'd be a bit weird, too..
Wouldn't that cause deadlocks if one session has VACUUM (SKIP_LOCKED)
and another session running vacuum or autovacuum if simply ignored?
That looks dangerous to me than simply skipping a lock that cannot be
taken.
--
Michael
On 3/30/18, 7:19 PM, "Andres Freund" <andres@anarazel.de> wrote:
Could you post a rebased version of the patch, with an incremental
addition of what you propose in a separate patch?
Here is a new patch set. I have attempted to add skip-locked
functionality everywhere it is needed, including VACUUM FULL.
On 4/3/18, 12:12 AM, "Michael Paquier" <michael@paquier.xyz> wrote:
On Fri, Mar 30, 2018 at 05:19:46PM -0700, Andres Freund wrote:
On 2018-03-30 18:39:01 +0000, Bossart, Nathan wrote:
I think the most straightforward approach for fixing this is to add
skip-locked functionality in find_all_inheritors(),
find_inheritance_children(), and vac_open_indexes(), but I am curious
what others think.For vac_open_indexes() it may make the most sense to introduce a new
try_open_index which wraps on top of try_relation_open with checks on
the opened relation's relkind.
I looked into this, but I noticed that the analogous
try_relation_open() implements missing-ok for relation_open(). So, I
thought it might be confusing to have try_index_open() be a skip-
locked version of index_open().
The new tests are now passing as expected, but I am still doing some
manual testing. Since there is quite a bit of added complexity in
this new patch set, I will certainly not be surprised if this gets
moved to the next commitfest.
Nathan
Attachments:
v7-0001-Add-skip_locked-argument-to-find_inheritance_chil.patchapplication/octet-stream; name=v7-0001-Add-skip_locked-argument-to-find_inheritance_chil.patchDownload
From 2e8936a89e9bbc8bd25e611617b435212402bc2b Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Thu, 5 Apr 2018 16:45:49 +0000
Subject: [PATCH v7 01/12] Add skip_locked argument to
find_inheritance_children().
---
src/backend/catalog/partition.c | 2 +-
src/backend/catalog/pg_inherits.c | 26 +++++++++++++++++++++++---
src/backend/commands/lockcmds.c | 2 +-
src/backend/commands/tablecmds.c | 16 ++++++++--------
src/backend/commands/trigger.c | 2 +-
src/include/catalog/pg_inherits_fn.h | 3 ++-
6 files changed, 36 insertions(+), 15 deletions(-)
diff --git a/src/backend/catalog/partition.c b/src/backend/catalog/partition.c
index 39ee773d93..42453c5007 100644
--- a/src/backend/catalog/partition.c
+++ b/src/backend/catalog/partition.c
@@ -232,7 +232,7 @@ RelationBuildPartitionDesc(Relation rel)
PartitionRangeBound **rbounds = NULL;
/* Get partition oids from pg_inherits */
- inhoids = find_inheritance_children(RelationGetRelid(rel), NoLock);
+ inhoids = find_inheritance_children(RelationGetRelid(rel), NoLock, false, NULL);
/* Collect bound spec nodes in a list */
i = 0;
diff --git a/src/backend/catalog/pg_inherits.c b/src/backend/catalog/pg_inherits.c
index 5a5beb9273..17f8b58fba 100644
--- a/src/backend/catalog/pg_inherits.c
+++ b/src/backend/catalog/pg_inherits.c
@@ -52,9 +52,14 @@ typedef struct SeenRelsEntry
* given rel; caller should already have locked it). If lockmode is NoLock
* then no locks are acquired, but caller must beware of race conditions
* against possible DROPs of child relations.
+ *
+ * If skip_locked is true, any child relations that cannot be locked immediately
+ * without waiting are not added to the returned OID list, and skipped will be
+ * set to true if it is provided. Otherwise, skipped will be set to false if it
+ * is provided.
*/
List *
-find_inheritance_children(Oid parentrelId, LOCKMODE lockmode)
+find_inheritance_children(Oid parentrelId, LOCKMODE lockmode, bool skip_locked, bool *skipped)
{
List *list = NIL;
Relation relation;
@@ -66,13 +71,19 @@ find_inheritance_children(Oid parentrelId, LOCKMODE lockmode)
int maxoids,
numoids,
i;
+ bool did_skip = false;
/*
* Can skip the scan if pg_class shows the relation has never had a
* subclass.
*/
if (!has_subclass(parentrelId))
+ {
+ if (skipped != NULL)
+ *skipped = did_skip;
+
return NIL;
+ }
/*
* Scan pg_inherits and build a working array of subclass OIDs.
@@ -125,7 +136,13 @@ find_inheritance_children(Oid parentrelId, LOCKMODE lockmode)
if (lockmode != NoLock)
{
/* Get the lock to synchronize against concurrent drop */
- LockRelationOid(inhrelid, lockmode);
+ if (!skip_locked)
+ LockRelationOid(inhrelid, lockmode);
+ else if (!ConditionalLockRelationOid(inhrelid, lockmode))
+ {
+ did_skip = true;
+ continue;
+ }
/*
* Now that we have the lock, double-check to see if the relation
@@ -146,6 +163,9 @@ find_inheritance_children(Oid parentrelId, LOCKMODE lockmode)
pfree(oidarr);
+ if (skipped != NULL)
+ *skipped = did_skip;
+
return list;
}
@@ -200,7 +220,7 @@ find_all_inheritors(Oid parentrelId, LOCKMODE lockmode, List **numparents)
ListCell *lc;
/* Get the direct children of this rel */
- currentchildren = find_inheritance_children(currentrel, lockmode);
+ currentchildren = find_inheritance_children(currentrel, lockmode, false, NULL);
/*
* Add to the queue only those children not already seen. This avoids
diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c
index b247c0fe2e..1039159946 100644
--- a/src/backend/commands/lockcmds.c
+++ b/src/backend/commands/lockcmds.c
@@ -120,7 +120,7 @@ LockTableRecurse(Oid reloid, LOCKMODE lockmode, bool nowait, Oid userid)
List *children;
ListCell *lc;
- children = find_inheritance_children(reloid, NoLock);
+ children = find_inheritance_children(reloid, NoLock, false, NULL);
foreach(lc, children)
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ec2f9616ed..15017c5a62 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -2694,7 +2694,7 @@ renameatt_internal(Oid myrelid,
* expected_parents will only be 0 if we are not already recursing.
*/
if (expected_parents == 0 &&
- find_inheritance_children(myrelid, NoLock) != NIL)
+ find_inheritance_children(myrelid, NoLock, false, NULL) != NIL)
ereport(ERROR,
(errcode(ERRCODE_INVALID_TABLE_DEFINITION),
errmsg("inherited column \"%s\" must be renamed in child tables too",
@@ -2893,7 +2893,7 @@ rename_constraint_internal(Oid myrelid,
else
{
if (expected_parents == 0 &&
- find_inheritance_children(myrelid, NoLock) != NIL)
+ find_inheritance_children(myrelid, NoLock, false, NULL) != NIL)
ereport(ERROR,
(errcode(ERRCODE_INVALID_TABLE_DEFINITION),
errmsg("inherited constraint \"%s\" must be renamed in child tables too",
@@ -5354,7 +5354,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
if (colDef->identity &&
recurse &&
- find_inheritance_children(myrelid, NoLock) != NIL)
+ find_inheritance_children(myrelid, NoLock, false, NULL) != NIL)
ereport(ERROR,
(errcode(ERRCODE_INVALID_TABLE_DEFINITION),
errmsg("cannot recursively add identity column to table that has child tables")));
@@ -5597,7 +5597,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
* routines, we have to do this one level of recursion at a time; we can't
* use find_all_inheritors to do it in one pass.
*/
- children = find_inheritance_children(RelationGetRelid(rel), lockmode);
+ children = find_inheritance_children(RelationGetRelid(rel), lockmode, false, NULL);
/*
* If we are told not to recurse, there had better not be any child
@@ -6693,7 +6693,7 @@ ATExecDropColumn(List **wqueue, Relation rel, const char *colName,
* routines, we have to do this one level of recursion at a time; we can't
* use find_all_inheritors to do it in one pass.
*/
- children = find_inheritance_children(RelationGetRelid(rel), lockmode);
+ children = find_inheritance_children(RelationGetRelid(rel), lockmode, false, NULL);
if (children)
{
@@ -7144,7 +7144,7 @@ ATAddCheckConstraint(List **wqueue, AlteredTableInfo *tab, Relation rel,
* routines, we have to do this one level of recursion at a time; we can't
* use find_all_inheritors to do it in one pass.
*/
- children = find_inheritance_children(RelationGetRelid(rel), lockmode);
+ children = find_inheritance_children(RelationGetRelid(rel), lockmode, false, NULL);
/*
* Check if ONLY was specified with ALTER TABLE. If so, allow the
@@ -8831,7 +8831,7 @@ ATExecDropConstraint(Relation rel, const char *constrName,
* use find_all_inheritors to do it in one pass.
*/
if (!is_no_inherit_constraint)
- children = find_inheritance_children(RelationGetRelid(rel), lockmode);
+ children = find_inheritance_children(RelationGetRelid(rel), lockmode, false, NULL);
else
children = NIL;
@@ -9173,7 +9173,7 @@ ATPrepAlterColumnType(List **wqueue,
}
}
else if (!recursing &&
- find_inheritance_children(RelationGetRelid(rel), NoLock) != NIL)
+ find_inheritance_children(RelationGetRelid(rel), NoLock, false, NULL) != NIL)
ereport(ERROR,
(errcode(ERRCODE_INVALID_TABLE_DEFINITION),
errmsg("type of inherited column \"%s\" must be changed in child tables too",
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index a189356cad..a7e549e0ae 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -1115,7 +1115,7 @@ CreateTrigger(CreateTrigStmt *stmt, const char *queryString,
ListCell *l;
List *idxs = NIL;
- idxs = find_inheritance_children(indexOid, ShareRowExclusiveLock);
+ idxs = find_inheritance_children(indexOid, ShareRowExclusiveLock, false, NULL);
foreach(l, idxs)
childTbls = lappend_oid(childTbls,
IndexGetRelation(lfirst_oid(l),
diff --git a/src/include/catalog/pg_inherits_fn.h b/src/include/catalog/pg_inherits_fn.h
index eebee977a5..34d2a2f39f 100644
--- a/src/include/catalog/pg_inherits_fn.h
+++ b/src/include/catalog/pg_inherits_fn.h
@@ -17,7 +17,8 @@
#include "nodes/pg_list.h"
#include "storage/lock.h"
-extern List *find_inheritance_children(Oid parentrelId, LOCKMODE lockmode);
+extern List *find_inheritance_children(Oid parentrelId, LOCKMODE lockmode,
+ bool skip_locked, bool *skipped);
extern List *find_all_inheritors(Oid parentrelId, LOCKMODE lockmode,
List **parents);
extern bool has_subclass(Oid relationId);
--
2.16.2
v7-0002-Add-skip_locked-argument-to-find_all_inheritors.patchapplication/octet-stream; name=v7-0002-Add-skip_locked-argument-to-find_all_inheritors.patchDownload
From a970360bc97f5d59b4d6919700432ae7d2573c73 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Thu, 5 Apr 2018 16:52:56 +0000
Subject: [PATCH v7 02/12] Add skip_locked argument to find_all_inheritors().
---
contrib/sepgsql/dml.c | 2 +-
src/backend/catalog/partition.c | 2 +-
src/backend/catalog/pg_inherits.c | 30 +++++++++++++++++++++++++++---
src/backend/commands/analyze.c | 2 +-
src/backend/commands/publicationcmds.c | 2 +-
src/backend/commands/tablecmds.c | 18 +++++++++---------
src/backend/commands/trigger.c | 2 +-
src/backend/commands/vacuum.c | 2 +-
src/backend/executor/execPartition.c | 2 +-
src/backend/optimizer/prep/prepunion.c | 2 +-
src/backend/tcop/utility.c | 2 +-
src/include/catalog/pg_inherits_fn.h | 2 +-
12 files changed, 46 insertions(+), 22 deletions(-)
diff --git a/contrib/sepgsql/dml.c b/contrib/sepgsql/dml.c
index c1fa320eb4..bbc72a59a2 100644
--- a/contrib/sepgsql/dml.c
+++ b/contrib/sepgsql/dml.c
@@ -330,7 +330,7 @@ sepgsql_dml_privileges(List *rangeTabls, bool abort_on_violation)
if (!rte->inh)
tableIds = list_make1_oid(rte->relid);
else
- tableIds = find_all_inheritors(rte->relid, NoLock, NULL);
+ tableIds = find_all_inheritors(rte->relid, NoLock, NULL, false);
foreach(li, tableIds)
{
diff --git a/src/backend/catalog/partition.c b/src/backend/catalog/partition.c
index 42453c5007..b701073dcb 100644
--- a/src/backend/catalog/partition.c
+++ b/src/backend/catalog/partition.c
@@ -1270,7 +1270,7 @@ check_default_allows_bound(Relation parent, Relation default_rel,
*/
if (default_rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
all_parts = find_all_inheritors(RelationGetRelid(default_rel),
- AccessExclusiveLock, NULL);
+ AccessExclusiveLock, NULL, false);
else
all_parts = list_make1_oid(RelationGetRelid(default_rel));
diff --git a/src/backend/catalog/pg_inherits.c b/src/backend/catalog/pg_inherits.c
index 17f8b58fba..c986540974 100644
--- a/src/backend/catalog/pg_inherits.c
+++ b/src/backend/catalog/pg_inherits.c
@@ -181,10 +181,12 @@ find_inheritance_children(Oid parentrelId, LOCKMODE lockmode, bool skip_locked,
* The specified lock type is acquired on all child relations (but not on the
* given rel; caller should already have locked it). If lockmode is NoLock
* then no locks are acquired, but caller must beware of race conditions
- * against possible DROPs of child relations.
+ * against possible DROPs of child relations. If skip_locked is true and a
+ * child relation can not be locked immediately without waiting, both the
+ * returned OID list and *numparents will be set to NIL.
*/
List *
-find_all_inheritors(Oid parentrelId, LOCKMODE lockmode, List **numparents)
+find_all_inheritors(Oid parentrelId, LOCKMODE lockmode, List **numparents, bool skip_locked)
{
/* hash table for O(1) rel_oid -> rel_numparents cell lookup */
HTAB *seen_rels;
@@ -192,6 +194,7 @@ find_all_inheritors(Oid parentrelId, LOCKMODE lockmode, List **numparents)
List *rels_list,
*rel_numparents;
ListCell *l;
+ bool skipped = false;
memset(&ctl, 0, sizeof(ctl));
ctl.keysize = sizeof(Oid);
@@ -220,7 +223,7 @@ find_all_inheritors(Oid parentrelId, LOCKMODE lockmode, List **numparents)
ListCell *lc;
/* Get the direct children of this rel */
- currentchildren = find_inheritance_children(currentrel, lockmode, false, NULL);
+ currentchildren = find_inheritance_children(currentrel, lockmode, skip_locked, &skipped);
/*
* Add to the queue only those children not already seen. This avoids
@@ -249,6 +252,27 @@ find_all_inheritors(Oid parentrelId, LOCKMODE lockmode, List **numparents)
hash_entry->numparents_cell = rel_numparents->tail;
}
}
+
+ if (skipped)
+ break;
+ }
+
+ /* if a child relation was skipped, set the return lists to NIL */
+ if (skipped)
+ {
+ /* release all locks */
+ foreach(l, rels_list)
+ {
+ Oid currentrel = lfirst_oid(l);
+
+ if (currentrel != parentrelId)
+ UnlockRelationOid(currentrel, lockmode);
+ }
+
+ list_free(rel_numparents);
+ list_free(rels_list);
+ rel_numparents = NIL;
+ rels_list = NIL;
}
if (numparents)
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 378784a93c..451e923132 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -1335,7 +1335,7 @@ acquire_inherited_sample_rows(Relation onerel, int elevel,
* the children.
*/
tableOIDs =
- find_all_inheritors(RelationGetRelid(onerel), AccessShareLock, NULL);
+ find_all_inheritors(RelationGetRelid(onerel), AccessShareLock, NULL, false);
/*
* Check that there's at least one descendant, else fail. This could
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 9c5aa9ebc2..b347d671a5 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -516,7 +516,7 @@ OpenTableList(List *tables)
List *children;
children = find_all_inheritors(myrelid, ShareUpdateExclusiveLock,
- NULL);
+ NULL, false);
foreach(child, children)
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 15017c5a62..3d19dd728f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1356,7 +1356,7 @@ ExecuteTruncate(TruncateStmt *stmt)
ListCell *child;
List *children;
- children = find_all_inheritors(myrelid, AccessExclusiveLock, NULL);
+ children = find_all_inheritors(myrelid, AccessExclusiveLock, NULL, false);
foreach(child, children)
{
@@ -2667,7 +2667,7 @@ renameatt_internal(Oid myrelid,
* outside the inheritance hierarchy being processed.
*/
child_oids = find_all_inheritors(myrelid, AccessExclusiveLock,
- &child_numparents);
+ &child_numparents, false);
/*
* find_all_inheritors does the recursive search of the inheritance
@@ -2877,7 +2877,7 @@ rename_constraint_internal(Oid myrelid,
*li;
child_oids = find_all_inheritors(myrelid, AccessExclusiveLock,
- &child_numparents);
+ &child_numparents, false);
forboth(lo, child_oids, li, child_numparents)
{
@@ -4943,7 +4943,7 @@ ATSimpleRecursion(List **wqueue, Relation rel,
ListCell *child;
List *children;
- children = find_all_inheritors(relid, lockmode, NULL);
+ children = find_all_inheritors(relid, lockmode, NULL, false);
/*
* find_all_inheritors does the recursive search of the inheritance
@@ -7918,7 +7918,7 @@ ATExecValidateConstraint(Relation rel, char *constrName, bool recurse,
*/
if (!recursing && !con->connoinherit)
children = find_all_inheritors(RelationGetRelid(rel),
- lockmode, NULL);
+ lockmode, NULL, false);
/*
* For CHECK constraints, we must ensure that we only mark the
@@ -9122,7 +9122,7 @@ ATPrepAlterColumnType(List **wqueue,
ListCell *child;
List *children;
- children = find_all_inheritors(relid, lockmode, NULL);
+ children = find_all_inheritors(relid, lockmode, NULL, false);
/*
* find_all_inheritors does the recursive search of the inheritance
@@ -11355,7 +11355,7 @@ ATExecAddInherit(Relation child_rel, RangeVar *parent, LOCKMODE lockmode)
* We use weakest lock we can on child's children, namely AccessShareLock.
*/
children = find_all_inheritors(RelationGetRelid(child_rel),
- AccessShareLock, NULL);
+ AccessShareLock, NULL, false);
if (list_member_oid(children, RelationGetRelid(parent_rel)))
ereport(ERROR,
@@ -14059,7 +14059,7 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
* weaker lock now and the stronger one only when needed.
*/
attachrel_children = find_all_inheritors(RelationGetRelid(attachrel),
- AccessExclusiveLock, NULL);
+ AccessExclusiveLock, NULL, false);
if (list_member_oid(attachrel_children, RelationGetRelid(rel)))
ereport(ERROR,
(errcode(ERRCODE_DUPLICATE_TABLE),
@@ -14252,7 +14252,7 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
get_proposed_default_constraint(partBoundConstraint);
defaultrel_children =
find_all_inheritors(defaultPartOid,
- AccessExclusiveLock, NULL);
+ AccessExclusiveLock, NULL, false);
ValidatePartitionConstraints(wqueue, defaultrel,
defaultrel_children,
defPartConstraint, true);
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index a7e549e0ae..1aa65ffc97 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -371,7 +371,7 @@ CreateTrigger(CreateTrigStmt *stmt, const char *queryString,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE;
if (partition_recurse)
list_free(find_all_inheritors(RelationGetRelid(rel),
- ShareRowExclusiveLock, NULL));
+ ShareRowExclusiveLock, NULL, false));
/* Compute tgtype */
TRIGGER_CLEAR_TYPE(tgtype);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index a1782c2874..db1b6e2d26 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -479,7 +479,7 @@ expand_vacuum_rel(VacuumRelation *vrel)
*/
if (include_parts)
{
- List *part_oids = find_all_inheritors(relid, NoLock, NULL);
+ List *part_oids = find_all_inheritors(relid, NoLock, NULL, false);
ListCell *part_lc;
foreach(part_lc, part_oids)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index ad532773a3..bf77eedbcb 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -77,7 +77,7 @@ ExecSetupPartitionTupleRouting(ModifyTableState *mtstate, Relation rel)
* Get the information about the partition tree after locking all the
* partitions.
*/
- (void) find_all_inheritors(RelationGetRelid(rel), RowExclusiveLock, NULL);
+ (void) find_all_inheritors(RelationGetRelid(rel), RowExclusiveLock, NULL, false);
proute = (PartitionTupleRouting *) palloc0(sizeof(PartitionTupleRouting));
proute->partition_dispatch_info =
RelationGetPartitionDispatchInfo(rel, &proute->num_dispatch,
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index 5236ab378e..d5112a6ec8 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -1556,7 +1556,7 @@ expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry *rte, Index rti)
lockmode = AccessShareLock;
/* Scan for all members of inheritance set, acquire needed locks */
- inhOIDs = find_all_inheritors(parentOID, lockmode, NULL);
+ inhOIDs = find_all_inheritors(parentOID, lockmode, NULL, false);
/*
* Check that there's at least one descendant, else treat as no-child
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 84f2591736..af4e1fb0ff 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1323,7 +1323,7 @@ ProcessUtilitySlow(ParseState *pstate,
rel = heap_open(relid, NoLock);
if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
inheritors = find_all_inheritors(relid, lockmode,
- NULL);
+ NULL, false);
heap_close(rel, NoLock);
}
diff --git a/src/include/catalog/pg_inherits_fn.h b/src/include/catalog/pg_inherits_fn.h
index 34d2a2f39f..5374dc48da 100644
--- a/src/include/catalog/pg_inherits_fn.h
+++ b/src/include/catalog/pg_inherits_fn.h
@@ -20,7 +20,7 @@
extern List *find_inheritance_children(Oid parentrelId, LOCKMODE lockmode,
bool skip_locked, bool *skipped);
extern List *find_all_inheritors(Oid parentrelId, LOCKMODE lockmode,
- List **parents);
+ List **parents, bool skip_locked);
extern bool has_subclass(Oid relationId);
extern bool has_superclass(Oid relationId);
extern bool typeInheritsFrom(Oid subclassTypeId, Oid superclassTypeId);
--
2.16.2
v7-0003-Add-skip_locked-argument-to-vac_open_indexes.patchapplication/octet-stream; name=v7-0003-Add-skip_locked-argument-to-vac_open_indexes.patchDownload
From ac8708a0f9c65f7cee4898aa8d83d09294e2fa0d Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Thu, 5 Apr 2018 16:58:27 +0000
Subject: [PATCH v7 03/12] Add skip_locked argument to vac_open_indexes().
---
src/backend/commands/analyze.c | 2 +-
src/backend/commands/vacuum.c | 27 +++++++++++++++++++++++++--
src/backend/commands/vacuumlazy.c | 2 +-
src/include/commands/vacuum.h | 3 ++-
4 files changed, 29 insertions(+), 5 deletions(-)
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 451e923132..5fdbbd630e 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -468,7 +468,7 @@ do_analyze_rel(Relation onerel, int options, VacuumParams *params,
* all.
*/
if (!inh)
- vac_open_indexes(onerel, AccessShareLock, &nindexes, &Irel);
+ vac_open_indexes(onerel, AccessShareLock, &nindexes, &Irel, false, NULL);
else
{
Irel = NULL;
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index db1b6e2d26..3ccf6cf98d 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1603,14 +1603,19 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
* vacuum even if the index isn't indisvalid; this is important because in a
* unique index, uniqueness checks will be performed anyway and had better not
* hit dangling index pointers.
+ *
+ * If skip_locked is true and an index cannot be locked immediately without
+ * waiting, *Irel will be set to NULL and nindexes will be set to 0. If skipped
+ * is provided, it will be set to true if an index was skipped (false otherwise).
*/
void
vac_open_indexes(Relation relation, LOCKMODE lockmode,
- int *nindexes, Relation **Irel)
+ int *nindexes, Relation **Irel, bool skip_locked, bool *skipped)
{
List *indexoidlist;
ListCell *indexoidscan;
int i;
+ bool did_skip = false;
Assert(lockmode != NoLock);
@@ -1631,13 +1636,31 @@ vac_open_indexes(Relation relation, LOCKMODE lockmode,
Oid indexoid = lfirst_oid(indexoidscan);
Relation indrel;
- indrel = index_open(indexoid, lockmode);
+ if (skip_locked && !ConditionalLockRelationOid(indexoid, lockmode))
+ {
+ did_skip = true;
+ break;
+ }
+
+ indrel = index_open(indexoid, skip_locked ? NoLock : lockmode);
if (IndexIsReady(indrel->rd_index))
(*Irel)[i++] = indrel;
else
index_close(indrel, lockmode);
}
+ if (did_skip)
+ {
+ vac_close_indexes(i, *Irel, lockmode);
+
+ /* clear all return values */
+ *Irel = NULL;
+ i = 0;
+ }
+
+ if (skipped != NULL)
+ *skipped = did_skip;
+
*nindexes = i;
list_free(indexoidlist);
diff --git a/src/backend/commands/vacuumlazy.c b/src/backend/commands/vacuumlazy.c
index d2a006671a..13e5bd452f 100644
--- a/src/backend/commands/vacuumlazy.c
+++ b/src/backend/commands/vacuumlazy.c
@@ -259,7 +259,7 @@ lazy_vacuum_rel(Relation onerel, int options, VacuumParams *params,
vacrelstats->lock_waiter_detected = false;
/* Open all indexes of the relation */
- vac_open_indexes(onerel, RowExclusiveLock, &nindexes, &Irel);
+ vac_open_indexes(onerel, RowExclusiveLock, &nindexes, &Irel, false, NULL);
vacrelstats->hasindex = (nindexes > 0);
/* Do the vacuuming */
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 85d472f0a5..69b1e384ef 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -160,7 +160,8 @@ extern void ExecVacuum(VacuumStmt *vacstmt, bool isTopLevel);
extern void vacuum(int options, List *relations, VacuumParams *params,
BufferAccessStrategy bstrategy, bool isTopLevel);
extern void vac_open_indexes(Relation relation, LOCKMODE lockmode,
- int *nindexes, Relation **Irel);
+ int *nindexes, Relation **Irel,
+ bool skip_locked, bool *skipped);
extern void vac_close_indexes(int nindexes, Relation *Irel, LOCKMODE lockmode);
extern double vac_estimate_reltuples(Relation relation,
BlockNumber total_pages,
--
2.16.2
v7-0004-Rename-VACOPT_NOWAIT-to-VACOPT_SKIP_LOCKED.patchapplication/octet-stream; name=v7-0004-Rename-VACOPT_NOWAIT-to-VACOPT_SKIP_LOCKED.patchDownload
From c8357942dedb0df2eade3e460189e8fd23bbecac Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Wed, 4 Apr 2018 16:54:53 +0000
Subject: [PATCH v7 04/12] Rename VACOPT_NOWAIT to VACOPT_SKIP_LOCKED.
---
src/backend/commands/analyze.c | 2 +-
src/backend/commands/vacuum.c | 2 +-
src/backend/postmaster/autovacuum.c | 2 +-
src/include/nodes/parsenodes.h | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 5fdbbd630e..4b05cb7fb3 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -143,7 +143,7 @@ analyze_rel(Oid relid, RangeVar *relation, int options,
* matter if we ever try to accumulate stats on dead tuples.) If the rel
* has been dropped since we last saw it, we don't need to process it.
*/
- if (!(options & VACOPT_NOWAIT))
+ if (!(options & VACOPT_SKIP_LOCKED))
onerel = try_relation_open(relid, ShareUpdateExclusiveLock);
else if (ConditionalLockRelationOid(relid, ShareUpdateExclusiveLock))
onerel = try_relation_open(relid, NoLock);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 3ccf6cf98d..7a48286108 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1371,7 +1371,7 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
* If we've been asked not to wait for the relation lock, acquire it first
* in non-blocking mode, before calling try_relation_open().
*/
- if (!(options & VACOPT_NOWAIT))
+ if (!(options & VACOPT_SKIP_LOCKED))
onerel = try_relation_open(relid, lmode);
else if (ConditionalLockRelationOid(relid, lmode))
onerel = try_relation_open(relid, NoLock);
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 3b90b16dae..2d1930e838 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -2904,7 +2904,7 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
tab->at_vacoptions = VACOPT_SKIPTOAST |
(dovacuum ? VACOPT_VACUUM : 0) |
(doanalyze ? VACOPT_ANALYZE : 0) |
- (!wraparound ? VACOPT_NOWAIT : 0);
+ (!wraparound ? VACOPT_SKIP_LOCKED : 0);
tab->at_params.freeze_min_age = freeze_min_age;
tab->at_params.freeze_table_age = freeze_table_age;
tab->at_params.multixact_freeze_min_age = multixact_freeze_min_age;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 699fa77bc7..5340d29f0b 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3158,7 +3158,7 @@ typedef enum VacuumOption
VACOPT_VERBOSE = 1 << 2, /* print progress info */
VACOPT_FREEZE = 1 << 3, /* FREEZE option */
VACOPT_FULL = 1 << 4, /* FULL (non-concurrent) vacuum */
- VACOPT_NOWAIT = 1 << 5, /* don't wait to get lock (autovacuum only) */
+ VACOPT_SKIP_LOCKED = 1 << 5, /* don't wait to get lock (autovacuum only) */
VACOPT_SKIPTOAST = 1 << 6, /* don't process the TOAST table, if any */
VACOPT_DISABLE_PAGE_SKIPPING = 1 << 7 /* don't skip any pages */
} VacuumOption;
--
2.16.2
v7-0005-Add-assertion-that-we-are-not-an-autovacuum-worke.patchapplication/octet-stream; name=v7-0005-Add-assertion-that-we-are-not-an-autovacuum-worke.patchDownload
From d1df6f5ea08cc0a0b166bd6913a05d208de55585 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Wed, 4 Apr 2018 16:55:33 +0000
Subject: [PATCH v7 05/12] Add assertion that we are not an autovacuum worker
in expand_vacuum_rel().
---
src/backend/commands/vacuum.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 7a48286108..55999b21d0 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -443,6 +443,12 @@ expand_vacuum_rel(VacuumRelation *vrel)
Form_pg_class classForm;
bool include_parts;
+ /*
+ * Since autovacuum workers supply OIDs when calling vacuum(), no
+ * autovacuum worker should reach this code.
+ */
+ Assert(!IsAutoVacuumWorkerProcess());
+
/*
* We transiently take AccessShareLock to protect the syscache lookup
* below, as well as find_all_inheritors's expectation that the caller
--
2.16.2
v7-0006-Create-a-helper-function-for-determining-the-log-.patchapplication/octet-stream; name=v7-0006-Create-a-helper-function-for-determining-the-log-.patchDownload
From 8f2eb87abc413586af1b3833aeb1eebf482f548b Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Thu, 5 Apr 2018 17:20:42 +0000
Subject: [PATCH v7 06/12] Create a helper function for determining the log
level for skipped relations.
---
src/backend/commands/analyze.c | 45 +++++++++++-----------------------
src/backend/commands/vacuum.c | 55 +++++++++++++++++++++++++-----------------
src/include/commands/vacuum.h | 1 +
3 files changed, 48 insertions(+), 53 deletions(-)
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 4b05cb7fb3..d9e31606ff 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -159,38 +159,21 @@ analyze_rel(Oid relid, RangeVar *relation, int options,
*/
if (!onerel)
{
- /*
- * If the RangeVar is not defined, we do not have enough information
- * to provide a meaningful log statement. Chances are that
- * analyze_rel's caller has intentionally not provided this
- * information so that this logging is skipped, anyway.
- */
- if (relation == NULL)
- return;
-
- /*
- * Determine the log level. For autovacuum logs, we emit a LOG if
- * log_autovacuum_min_duration is not disabled. For manual ANALYZE,
- * we emit a WARNING to match the log statements in the permissions
- * checks.
- */
- if (!IsAutoVacuumWorkerProcess())
- elevel = WARNING;
- else if (params->log_min_duration >= 0)
- elevel = LOG;
- else
- return;
+ elevel = get_elevel_for_skipped_relation(relation, params);
- if (!rel_lock)
- ereport(elevel,
- (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
- errmsg("skipping analyze of \"%s\" --- lock not available",
- relation->relname)));
- else
- ereport(elevel,
- (errcode(ERRCODE_UNDEFINED_TABLE),
- errmsg("skipping analyze of \"%s\" --- relation no longer exists",
- relation->relname)));
+ if (elevel != 0)
+ {
+ if (!rel_lock)
+ ereport(elevel,
+ (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
+ errmsg("skipping analyze of \"%s\" --- lock not available",
+ relation->relname)));
+ else
+ ereport(elevel,
+ (errcode(ERRCODE_UNDEFINED_TABLE),
+ errmsg("skipping analyze of \"%s\" --- relation no longer exists",
+ relation->relname)));
+ }
return;
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 55999b21d0..d7967dda6b 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1393,28 +1393,7 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
*/
if (!onerel)
{
- int elevel = 0;
-
- /*
- * Determine the log level.
- *
- * If the RangeVar is not defined, we do not have enough information
- * to provide a meaningful log statement. Chances are that
- * vacuum_rel's caller has intentionally not provided this information
- * so that this logging is skipped, anyway.
- *
- * Otherwise, for autovacuum logs, we emit a LOG if
- * log_autovacuum_min_duration is not disabled. For manual VACUUM, we
- * emit a WARNING to match the log statements in the permission
- * checks.
- */
- if (relation != NULL)
- {
- if (!IsAutoVacuumWorkerProcess())
- elevel = WARNING;
- else if (params->log_min_duration >= 0)
- elevel = LOG;
- }
+ int elevel = get_elevel_for_skipped_relation(relation, params);
if (elevel != 0)
{
@@ -1724,3 +1703,35 @@ vacuum_delay_point(void)
CHECK_FOR_INTERRUPTS();
}
}
+
+/*
+ * get_elevel_for_skip_locked
+ *
+ * This is a helper function for determining the appropriate logging level for
+ * messages related to SKIP LOCKED or concurrently dropped relations.
+ *
+ * If the RangeVar is not defined, we do not have enough information to provide
+ * a meaningful log statement, and 0 is returned. Chances are that the caller
+ * has intentionally not provided this information so that the logging is
+ * skipped anyway.
+ *
+ * Otherwise, for autovacuum logs, we emit a LOG if log_autovacuum_min_duration
+ * is not disabled. For manual commands, we emit a WARNING to match the log
+ * statements in the permission checks for VACUUM and ANALYZE.
+ */
+int
+get_elevel_for_skipped_relation(RangeVar *relation, VacuumParams *params)
+{
+ Assert(params != NULL);
+
+ if (relation == NULL)
+ return 0;
+
+ if (!IsAutoVacuumWorkerProcess())
+ return WARNING;
+
+ if (params->log_min_duration >= 0)
+ return LOG;
+
+ return 0;
+}
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 69b1e384ef..64215ea60a 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -186,6 +186,7 @@ extern void vacuum_set_xid_limits(Relation rel,
MultiXactId *mxactFullScanLimit);
extern void vac_update_datfrozenxid(void);
extern void vacuum_delay_point(void);
+extern int get_elevel_for_skipped_relation(RangeVar *relation, VacuumParams *params);
/* in commands/vacuumlazy.c */
extern void lazy_vacuum_rel(Relation onerel, int options,
--
2.16.2
v7-0007-Create-a-helper-function-for-cleaning-up-from-do_.patchapplication/octet-stream; name=v7-0007-Create-a-helper-function-for-cleaning-up-from-do_.patchDownload
From 6ef01699d703389b69d43a1894b116ec611b8e32 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Thu, 5 Apr 2018 17:37:54 +0000
Subject: [PATCH v7 07/12] Create a helper function for cleaning up from
do_analyze_rel().
---
src/backend/commands/analyze.c | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index d9e31606ff..03e0bad1f3 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -102,6 +102,9 @@ static void update_attstats(Oid relid, bool inh,
int natts, VacAttrStats **vacattrstats);
static Datum std_fetch_func(VacAttrStatsP stats, int rownum, bool *isNull);
static Datum ind_fetch_func(VacAttrStatsP stats, int rownum, bool *isNull);
+static void do_analyze_rel_cleanup(int save_nestlevel, Oid save_userid,
+ int save_sec_context, MemoryContext caller_context,
+ RangeVar *rangeVar, VacuumParams *params);
/*
@@ -705,6 +708,29 @@ do_analyze_rel(Relation onerel, int options, VacuumParams *params,
pg_rusage_show(&ru0))));
}
+ /* pass NULL for the RangeVar to avoid the SKIP LOCKED message */
+ do_analyze_rel_cleanup(save_nestlevel, save_userid, save_sec_context,
+ caller_context, NULL, params);
+}
+
+/*
+ * Convenience function for cleaning up before exiting do_analyze_rel().
+ *
+ * This also emits a "skipping analyze" log statement based on the rangeVar and
+ * params provided. If rangeVar is NULL, no such log statement will be emitted.
+ */
+static void
+do_analyze_rel_cleanup(int save_nestlevel, Oid save_userid, int save_sec_context,
+ MemoryContext caller_context, RangeVar *rangeVar, VacuumParams *params)
+{
+ int elevel = get_elevel_for_skipped_relation(rangeVar, params);
+
+ if (elevel != 0)
+ ereport(elevel,
+ (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
+ errmsg("skipping analyze of \"%s\" --- lock not available",
+ rangeVar->relname)));
+
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
--
2.16.2
v7-0008-Skip-VACUUM-ANALYZE-with-VACOPT_SKIP_LOCKED-if-in.patchapplication/octet-stream; name=v7-0008-Skip-VACUUM-ANALYZE-with-VACOPT_SKIP_LOCKED-if-in.patchDownload
From 6f29356c60a6f44c4f2eaa56cf1e32b9c4f25524 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Thu, 5 Apr 2018 19:00:40 +0000
Subject: [PATCH v7 08/12] Skip VACUUM/ANALYZE with VACOPT_SKIP_LOCKED if
indexes can't be opened.
---
src/backend/commands/analyze.c | 29 ++++++++++++++++++++++++-----
src/backend/commands/vacuum.c | 23 +++++++++++++++++++----
src/backend/commands/vacuumlazy.c | 23 +++++++++++++++++++++--
src/include/commands/vacuum.h | 2 +-
4 files changed, 65 insertions(+), 12 deletions(-)
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 03e0bad1f3..0421d77ad7 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -84,7 +84,7 @@ static BufferAccessStrategy vac_strategy;
static void do_analyze_rel(Relation onerel, int options,
VacuumParams *params, List *va_cols,
AcquireSampleRowsFunc acquirefunc, BlockNumber relpages,
- bool inh, bool in_outer_xact, int elevel);
+ bool inh, bool in_outer_xact, int elevel, RangeVar *rangeVar);
static void compute_index_stats(Relation onerel, double totalrows,
AnlIndexData *indexdata, int nindexes,
HeapTuple *rows, int numrows,
@@ -294,14 +294,14 @@ analyze_rel(Oid relid, RangeVar *relation, int options,
*/
if (onerel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
do_analyze_rel(onerel, options, params, va_cols, acquirefunc,
- relpages, false, in_outer_xact, elevel);
+ relpages, false, in_outer_xact, elevel, relation);
/*
* If there are child tables, do recursive ANALYZE.
*/
if (onerel->rd_rel->relhassubclass)
do_analyze_rel(onerel, options, params, va_cols, acquirefunc, relpages,
- true, in_outer_xact, elevel);
+ true, in_outer_xact, elevel, relation);
/*
* Close source relation now, but keep lock so that no one deletes it
@@ -331,7 +331,7 @@ static void
do_analyze_rel(Relation onerel, int options, VacuumParams *params,
List *va_cols, AcquireSampleRowsFunc acquirefunc,
BlockNumber relpages, bool inh, bool in_outer_xact,
- int elevel)
+ int elevel, RangeVar *rangeVar)
{
int attr_cnt,
tcnt,
@@ -454,7 +454,26 @@ do_analyze_rel(Relation onerel, int options, VacuumParams *params,
* all.
*/
if (!inh)
- vac_open_indexes(onerel, AccessShareLock, &nindexes, &Irel, false, NULL);
+ {
+ bool skipped = false;
+
+ vac_open_indexes(onerel, AccessShareLock, &nindexes, &Irel,
+ (options & VACOPT_SKIP_LOCKED), &skipped);
+
+ /*
+ * If SKIP LOCKED is specified and we would have to wait for a lock on
+ * an index, skip analyzing the relation.
+ */
+ if (skipped)
+ {
+ do_analyze_rel_cleanup(save_nestlevel, save_userid,
+ save_sec_context, caller_context,
+ rangeVar, params);
+
+ return;
+ }
+
+ }
else
{
Irel = NULL;
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d7967dda6b..fb7dd8d536 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1314,6 +1314,7 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
int save_sec_context;
int save_nestlevel;
bool rel_lock = true;
+ bool skipped = false;
Assert(params != NULL);
@@ -1539,7 +1540,18 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
(options & VACOPT_VERBOSE) != 0);
}
else
- lazy_vacuum_rel(onerel, options, params, vac_strategy);
+ skipped = !lazy_vacuum_rel(onerel, options, params, vac_strategy);
+
+ if (skipped)
+ {
+ int elevel = get_elevel_for_skipped_relation(relation, params);
+
+ if (elevel != 0)
+ ereport(elevel,
+ (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
+ errmsg("skipping vacuum of \"%s\" --- lock not available",
+ relation->relname)));
+ }
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -1563,8 +1575,11 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
* "analyze" will not get done on the toast table. This is good, because
* the toaster always uses hardcoded index access and statistics are
* totally unimportant for toast relations.
+ *
+ * If we skipped vacuuming the main relation, skip vacuuming the secondary
+ * toast rel as well.
*/
- if (toast_relid != InvalidOid)
+ if (!skipped && toast_relid != InvalidOid)
vacuum_rel(toast_relid, NULL, options, params);
/*
@@ -1572,8 +1587,8 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
*/
UnlockRelationIdForSession(&onerelid, lmode);
- /* Report that we really did it. */
- return true;
+ /* Report if we really did it. */
+ return !skipped;
}
diff --git a/src/backend/commands/vacuumlazy.c b/src/backend/commands/vacuumlazy.c
index 13e5bd452f..3a584b5115 100644
--- a/src/backend/commands/vacuumlazy.c
+++ b/src/backend/commands/vacuumlazy.c
@@ -186,8 +186,12 @@ static bool heap_page_is_all_visible(Relation rel, Buffer buf,
*
* At entry, we have already established a transaction and opened
* and locked the relation.
+ *
+ * If SKIP LOCKED is specified in the options and lazy_vacuum_rel()
+ * skips the relation due to a conflicting lock, false is returned.
+ * Else, true is returned.
*/
-void
+bool
lazy_vacuum_rel(Relation onerel, int options, VacuumParams *params,
BufferAccessStrategy bstrategy)
{
@@ -209,6 +213,7 @@ lazy_vacuum_rel(Relation onerel, int options, VacuumParams *params,
double new_live_tuples;
TransactionId new_frozen_xid;
MultiXactId new_min_multi;
+ bool skipped = false;
Assert(params != NULL);
@@ -259,7 +264,19 @@ lazy_vacuum_rel(Relation onerel, int options, VacuumParams *params,
vacrelstats->lock_waiter_detected = false;
/* Open all indexes of the relation */
- vac_open_indexes(onerel, RowExclusiveLock, &nindexes, &Irel, false, NULL);
+ vac_open_indexes(onerel, RowExclusiveLock, &nindexes, &Irel,
+ (options & VACOPT_SKIP_LOCKED), &skipped);
+
+ /*
+ * If SKIP LOCKED is specified and we would have to wait for a lock on an
+ * index, skip vacuuming the relation.
+ */
+ if (skipped)
+ {
+ pgstat_progress_end_command();
+ return false;
+ }
+
vacrelstats->hasindex = (nindexes > 0);
/* Do the vacuuming */
@@ -409,6 +426,8 @@ lazy_vacuum_rel(Relation onerel, int options, VacuumParams *params,
pfree(buf.data);
}
}
+
+ return true;
}
/*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 64215ea60a..23fd8392c5 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -189,7 +189,7 @@ extern void vacuum_delay_point(void);
extern int get_elevel_for_skipped_relation(RangeVar *relation, VacuumParams *params);
/* in commands/vacuumlazy.c */
-extern void lazy_vacuum_rel(Relation onerel, int options,
+extern bool lazy_vacuum_rel(Relation onerel, int options,
VacuumParams *params, BufferAccessStrategy bstrategy);
/* in commands/analyze.c */
--
2.16.2
v7-0009-Skip-ANALYZE-with-VACOPT_SKIP_LOCKED-if-acquire_i.patchapplication/octet-stream; name=v7-0009-Skip-ANALYZE-with-VACOPT_SKIP_LOCKED-if-acquire_i.patchDownload
From 7ad5ee333bc8fee0c860be2ad537eeb5d5d728ce Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Thu, 5 Apr 2018 18:43:45 +0000
Subject: [PATCH v7 09/12] Skip ANALYZE with VACOPT_SKIP_LOCKED if
acquire_inherited_sample_rows() blocks.
---
src/backend/commands/analyze.c | 54 ++++++++++++++++++++++++++++++++++++------
1 file changed, 47 insertions(+), 7 deletions(-)
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 0421d77ad7..1997726fd1 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -97,7 +97,8 @@ static int acquire_sample_rows(Relation onerel, int elevel,
static int compare_rows(const void *a, const void *b);
static int acquire_inherited_sample_rows(Relation onerel, int elevel,
HeapTuple *rows, int targrows,
- double *totalrows, double *totaldeadrows);
+ double *totalrows, double *totaldeadrows,
+ bool skip_locked, bool *skipped);
static void update_attstats(Oid relid, bool inh,
int natts, VacAttrStats **vacattrstats);
static Datum std_fetch_func(VacAttrStatsP stats, int rownum, bool *isNull);
@@ -353,6 +354,7 @@ do_analyze_rel(Relation onerel, int options, VacuumParams *params,
Oid save_userid;
int save_sec_context;
int save_nestlevel;
+ bool skipped = false;
if (inh)
ereport(elevel,
@@ -550,9 +552,23 @@ do_analyze_rel(Relation onerel, int options, VacuumParams *params,
*/
rows = (HeapTuple *) palloc(targrows * sizeof(HeapTuple));
if (inh)
+ {
numrows = acquire_inherited_sample_rows(onerel, elevel,
rows, targrows,
- &totalrows, &totaldeadrows);
+ &totalrows, &totaldeadrows,
+ (options & VACOPT_SKIP_LOCKED), &skipped);
+
+ /*
+ * If SKIP LOCKED is specified and we would have to wait for a lock on
+ * a child relation, skip analyzing the relation.
+ */
+ if (skipped)
+ {
+ do_analyze_rel_cleanup(save_nestlevel, save_userid, save_sec_context,
+ caller_context, rangeVar, params);
+ return;
+ }
+ }
else
numrows = (*acquirefunc) (onerel, elevel,
rows, targrows,
@@ -1337,15 +1353,20 @@ compare_rows(const void *a, const void *b)
/*
* acquire_inherited_sample_rows -- acquire sample rows from inheritance tree
*
- * This has the same API as acquire_sample_rows, except that rows are
+ * This has roughly the same API as acquire_sample_rows, except that rows are
* collected from all inheritance children as well as the specified table.
- * We fail and return zero if there are no inheritance children, or if all
- * children are foreign tables that don't support ANALYZE.
+ * We fail and return zero if there are no inheritance children, if all
+ * children are foreign tables that don't support ANALYZE, or if skip_locked
+ * is true and we cannot obtain a lock on all children without waiting.
+ *
+ * If skipped is supplied, it will be set to true if skip_locked takes effect
+ * (false otherwise).
*/
static int
acquire_inherited_sample_rows(Relation onerel, int elevel,
HeapTuple *rows, int targrows,
- double *totalrows, double *totaldeadrows)
+ double *totalrows, double *totaldeadrows,
+ bool skip_locked, bool *skipped)
{
List *tableOIDs;
Relation *rels;
@@ -1363,7 +1384,26 @@ acquire_inherited_sample_rows(Relation onerel, int elevel,
* the children.
*/
tableOIDs =
- find_all_inheritors(RelationGetRelid(onerel), AccessShareLock, NULL, false);
+ find_all_inheritors(RelationGetRelid(onerel), AccessShareLock, NULL, skip_locked);
+
+ /*
+ * If tableOIDs is NIL, skip_locked took effect. In that case, we should
+ * fail, being sure to set skipped if it is provided.
+ */
+ if (tableOIDs == NIL)
+ {
+ if (skipped != NULL)
+ *skipped = true;
+
+ return 0;
+ }
+
+ /*
+ * All skip_locked-related logic precedes this point, so we can just set
+ * skipped to false if it is provided.
+ */
+ if (skipped != NULL)
+ *skipped = false;
/*
* Check that there's at least one descendant, else fail. This could
--
2.16.2
v7-0010-Skip-VACUUM-ANALYZE-with-VACOPT_SKIP_LOCKED-if-ex.patchapplication/octet-stream; name=v7-0010-Skip-VACUUM-ANALYZE-with-VACOPT_SKIP_LOCKED-if-ex.patchDownload
From 2a38290b6ccc36d3d0389d833aed44b7d1ac6d64 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Thu, 5 Apr 2018 18:52:11 +0000
Subject: [PATCH v7 10/12] Skip VACUUM/ANALYZE with VACOPT_SKIP_LOCKED if
expand_vacuum_rel() would block.
---
src/backend/commands/vacuum.c | 30 ++++++++++++++++++++++++++----
1 file changed, 26 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index fb7dd8d536..f19d112af6 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -68,7 +68,7 @@ static BufferAccessStrategy vac_strategy;
/* non-export function prototypes */
-static List *expand_vacuum_rel(VacuumRelation *vrel);
+static List *expand_vacuum_rel(VacuumRelation *vrel, int options);
static List *get_all_vacuum_rels(void);
static void vac_truncate_clog(TransactionId frozenXID,
MultiXactId minMulti,
@@ -257,7 +257,7 @@ vacuum(int options, List *relations, VacuumParams *params,
List *sublist;
MemoryContext old_context;
- sublist = expand_vacuum_rel(vrel);
+ sublist = expand_vacuum_rel(vrel, options);
old_context = MemoryContextSwitchTo(vac_context);
newrels = list_concat(newrels, sublist);
MemoryContextSwitchTo(old_context);
@@ -423,7 +423,7 @@ vacuum(int options, List *relations, VacuumParams *params,
* are made in vac_context.
*/
static List *
-expand_vacuum_rel(VacuumRelation *vrel)
+expand_vacuum_rel(VacuumRelation *vrel, int options)
{
List *vacrels = NIL;
MemoryContext oldcontext;
@@ -454,7 +454,29 @@ expand_vacuum_rel(VacuumRelation *vrel)
* below, as well as find_all_inheritors's expectation that the caller
* holds some lock on the starting relation.
*/
- relid = RangeVarGetRelid(vrel->relation, AccessShareLock, false);
+ relid = RangeVarGetRelidExtended(vrel->relation,
+ AccessShareLock,
+ (options & VACOPT_SKIP_LOCKED) ? RVR_SKIP_LOCKED : 0,
+ NULL, NULL);
+
+ /*
+ * If the lock is unavailable, emit the same log statement that
+ * vacuum_rel() and analyze_rel() would.
+ */
+ if (!OidIsValid(relid))
+ {
+ if (options & VACOPT_VACUUM)
+ ereport(WARNING,
+ (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
+ errmsg("skipping vacuum of \"%s\" --- lock not available",
+ vrel->relation->relname)));
+ else
+ ereport(WARNING,
+ (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
+ errmsg("skipping analyze of \"%s\" --- lock not available",
+ vrel->relation->relname)));
+ return vacrels;
+ }
/*
* Make a returnable VacuumRelation for this rel.
--
2.16.2
v7-0011-Skip-VACUUM-FULL-with-VACOPT_SKIP_LOCKED-if-clust.patchapplication/octet-stream; name=v7-0011-Skip-VACUUM-FULL-with-VACOPT_SKIP_LOCKED-if-clust.patchDownload
From 81f01e3cc649bd752944524dddc9382b90d8d0a5 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Thu, 5 Apr 2018 19:04:25 +0000
Subject: [PATCH v7 11/12] Skip VACUUM FULL with VACOPT_SKIP_LOCKED if
cluster_rel() would block.
---
src/backend/commands/cluster.c | 37 +++++++++++++++++++++++++------------
src/backend/commands/vacuum.c | 5 +++--
src/include/commands/cluster.h | 4 ++--
3 files changed, 30 insertions(+), 16 deletions(-)
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index d088dc11a6..63486bf34f 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -186,7 +186,7 @@ cluster(ClusterStmt *stmt, bool isTopLevel)
heap_close(rel, NoLock);
/* Do the job. */
- cluster_rel(tableOid, indexOid, false, stmt->verbose);
+ cluster_rel(tableOid, indexOid, false, stmt->verbose, false);
}
else
{
@@ -234,7 +234,7 @@ cluster(ClusterStmt *stmt, bool isTopLevel)
/* functions in indexes may want a snapshot set */
PushActiveSnapshot(GetTransactionSnapshot());
/* Do the job. */
- cluster_rel(rvtc->tableOid, rvtc->indexOid, true, stmt->verbose);
+ cluster_rel(rvtc->tableOid, rvtc->indexOid, true, stmt->verbose, false);
PopActiveSnapshot();
CommitTransactionCommand();
}
@@ -263,9 +263,15 @@ cluster(ClusterStmt *stmt, bool isTopLevel)
* If indexOid is InvalidOid, the table will be rewritten in physical order
* instead of index order. This is the new implementation of VACUUM FULL,
* and error messages should refer to the operation as VACUUM not CLUSTER.
+ *
+ * If skip_locked is true and the relation cannot be immediately locked,
+ * processing will be skipped and false will be returned (otherwise, true is
+ * returned). Note that skip_locked will not prevent blocking on the index
+ * provided. Since this functionality is only used for VACUUM FULL, this
+ * omission has no effect.
*/
-void
-cluster_rel(Oid tableOid, Oid indexOid, bool recheck, bool verbose)
+bool
+cluster_rel(Oid tableOid, Oid indexOid, bool recheck, bool verbose, bool skip_locked)
{
Relation OldHeap;
@@ -278,11 +284,16 @@ cluster_rel(Oid tableOid, Oid indexOid, bool recheck, bool verbose)
* case, since cluster() already did it.) The index lock is taken inside
* check_index_is_clusterable.
*/
- OldHeap = try_relation_open(tableOid, AccessExclusiveLock);
+ if (!skip_locked)
+ OldHeap = try_relation_open(tableOid, AccessExclusiveLock);
+ else if (ConditionalLockRelationOid(tableOid, AccessExclusiveLock))
+ OldHeap = try_relation_open(tableOid, NoLock);
+ else
+ return false;
/* If the table has gone away, we can skip processing it */
if (!OldHeap)
- return;
+ return true;
/*
* Since we may open a new transaction for each relation, we have to check
@@ -301,7 +312,7 @@ cluster_rel(Oid tableOid, Oid indexOid, bool recheck, bool verbose)
if (!pg_class_ownercheck(tableOid, GetUserId()))
{
relation_close(OldHeap, AccessExclusiveLock);
- return;
+ return true;
}
/*
@@ -315,7 +326,7 @@ cluster_rel(Oid tableOid, Oid indexOid, bool recheck, bool verbose)
if (RELATION_IS_OTHER_TEMP(OldHeap))
{
relation_close(OldHeap, AccessExclusiveLock);
- return;
+ return true;
}
if (OidIsValid(indexOid))
@@ -326,7 +337,7 @@ cluster_rel(Oid tableOid, Oid indexOid, bool recheck, bool verbose)
if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(indexOid)))
{
relation_close(OldHeap, AccessExclusiveLock);
- return;
+ return true;
}
/*
@@ -336,14 +347,14 @@ cluster_rel(Oid tableOid, Oid indexOid, bool recheck, bool verbose)
if (!HeapTupleIsValid(tuple)) /* probably can't happen */
{
relation_close(OldHeap, AccessExclusiveLock);
- return;
+ return true;
}
indexForm = (Form_pg_index) GETSTRUCT(tuple);
if (!indexForm->indisclustered)
{
ReleaseSysCache(tuple);
relation_close(OldHeap, AccessExclusiveLock);
- return;
+ return true;
}
ReleaseSysCache(tuple);
}
@@ -397,7 +408,7 @@ cluster_rel(Oid tableOid, Oid indexOid, bool recheck, bool verbose)
!RelationIsPopulated(OldHeap))
{
relation_close(OldHeap, AccessExclusiveLock);
- return;
+ return true;
}
/*
@@ -412,6 +423,8 @@ cluster_rel(Oid tableOid, Oid indexOid, bool recheck, bool verbose)
rebuild_relation(OldHeap, indexOid, verbose);
/* NB: rebuild_relation does heap_close() on OldHeap */
+
+ return true;
}
/*
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index f19d112af6..74349f30d9 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1558,8 +1558,9 @@ vacuum_rel(Oid relid, RangeVar *relation, int options, VacuumParams *params)
onerel = NULL;
/* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
- cluster_rel(relid, InvalidOid, false,
- (options & VACOPT_VERBOSE) != 0);
+ skipped = !cluster_rel(relid, InvalidOid, false,
+ (options & VACOPT_VERBOSE) != 0,
+ (options & VACOPT_SKIP_LOCKED) != 0);
}
else
skipped = !lazy_vacuum_rel(onerel, options, params, vac_strategy);
diff --git a/src/include/commands/cluster.h b/src/include/commands/cluster.h
index b338cb1098..49bb35fafe 100644
--- a/src/include/commands/cluster.h
+++ b/src/include/commands/cluster.h
@@ -19,8 +19,8 @@
extern void cluster(ClusterStmt *stmt, bool isTopLevel);
-extern void cluster_rel(Oid tableOid, Oid indexOid, bool recheck,
- bool verbose);
+extern bool cluster_rel(Oid tableOid, Oid indexOid, bool recheck,
+ bool verbose, bool skip_locked);
extern void check_index_is_clusterable(Relation OldHeap, Oid indexOid,
bool recheck, LOCKMODE lockmode);
extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal);
--
2.16.2
v7-0012-Open-up-VACOPT_SKIP_LOCKED-to-users.patchapplication/octet-stream; name=v7-0012-Open-up-VACOPT_SKIP_LOCKED-to-users.patchDownload
From ce073c5046c659476a250d2879dee5df0f7f78f6 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <bossartn@amazon.com>
Date: Wed, 4 Apr 2018 21:53:48 +0000
Subject: [PATCH v7 12/12] Open up VACOPT_SKIP_LOCKED to users.
---
doc/src/sgml/ref/analyze.sgml | 12 ++
doc/src/sgml/ref/vacuum.sgml | 12 ++
src/backend/parser/gram.y | 2 +
src/include/nodes/parsenodes.h | 2 +-
src/test/isolation/expected/vacuum-skip-locked.out | 171 +++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
src/test/isolation/specs/vacuum-skip-locked.spec | 59 +++++++
src/test/regress/expected/vacuum.out | 10 ++
src/test/regress/sql/vacuum.sql | 10 ++
9 files changed, 278 insertions(+), 1 deletion(-)
create mode 100644 src/test/isolation/expected/vacuum-skip-locked.out
create mode 100644 src/test/isolation/specs/vacuum-skip-locked.spec
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 10b3a9a673..4e6ae38a52 100644
--- a/doc/src/sgml/ref/analyze.sgml
+++ b/doc/src/sgml/ref/analyze.sgml
@@ -27,6 +27,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
VERBOSE
+ SKIP LOCKED
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -76,6 +77,17 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>SKIP LOCKED</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>ANALYZE</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index b760e8ede1..1ef5bc9d4a 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -31,6 +31,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
VERBOSE
ANALYZE
DISABLE_PAGE_SKIPPING
+ SKIP LOCKED
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -160,6 +161,17 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>SKIP LOCKED</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should not wait for any
+ conflicting locks to be released: if a relation cannot be locked
+ immediately without waiting, the relation is skipped.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><replaceable class="parameter">table_name</replaceable></term>
<listitem>
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 1592b58bb4..e170e8762a 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -10543,6 +10543,7 @@ vacuum_option_elem:
errmsg("unrecognized VACUUM option \"%s\"", $1),
parser_errposition(@1)));
}
+ | SKIP LOCKED { $$ = VACOPT_SKIP_LOCKED; }
;
AnalyzeStmt: analyze_keyword opt_verbose opt_vacuum_relation_list
@@ -10570,6 +10571,7 @@ analyze_option_list:
analyze_option_elem:
VERBOSE { $$ = VACOPT_VERBOSE; }
+ | SKIP LOCKED { $$ = VACOPT_SKIP_LOCKED; }
;
analyze_keyword:
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 5340d29f0b..8d9a57215a 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3158,7 +3158,7 @@ typedef enum VacuumOption
VACOPT_VERBOSE = 1 << 2, /* print progress info */
VACOPT_FREEZE = 1 << 3, /* FREEZE option */
VACOPT_FULL = 1 << 4, /* FULL (non-concurrent) vacuum */
- VACOPT_SKIP_LOCKED = 1 << 5, /* don't wait to get lock (autovacuum only) */
+ VACOPT_SKIP_LOCKED = 1 << 5, /* don't wait to get lock */
VACOPT_SKIPTOAST = 1 << 6, /* don't process the TOAST table, if any */
VACOPT_DISABLE_PAGE_SKIPPING = 1 << 7 /* don't skip any pages */
} VacuumOption;
diff --git a/src/test/isolation/expected/vacuum-skip-locked.out b/src/test/isolation/expected/vacuum-skip-locked.out
new file mode 100644
index 0000000000..a6fe617db1
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-skip-locked.out
@@ -0,0 +1,171 @@
+Parsed test spec with 2 sessions
+
+starting permutation: lock_share vac_specified commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_specified: VACUUM (SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share vac_all_parts commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step vac_all_parts: VACUUM (SKIP LOCKED) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share analyze_specified commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping analyze of "part1" --- lock not available
+step analyze_specified: ANALYZE (SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share analyze_all_parts commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step analyze_all_parts: ANALYZE (SKIP LOCKED) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share vac_analyze_specified commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_analyze_specified: VACUUM (ANALYZE, SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share vac_analyze_all_parts commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step vac_analyze_all_parts: VACUUM (ANALYZE, SKIP LOCKED) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share vac_full_specified commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_full_specified: VACUUM (SKIP LOCKED, FULL) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_share vac_full_all_parts commit
+step lock_share:
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+
+step vac_full_all_parts: VACUUM (SKIP LOCKED, FULL) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive vac_specified commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_specified: VACUUM (SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive vac_all_parts commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+step vac_all_parts: VACUUM (SKIP LOCKED) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive analyze_specified commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+WARNING: skipping analyze of "part1" --- lock not available
+step analyze_specified: ANALYZE (SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive analyze_all_parts commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+WARNING: skipping analyze of "parted" --- lock not available
+step analyze_all_parts: ANALYZE (SKIP LOCKED) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive vac_analyze_specified commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_analyze_specified: VACUUM (ANALYZE, SKIP LOCKED) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive vac_analyze_all_parts commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+WARNING: skipping analyze of "parted" --- lock not available
+step vac_analyze_all_parts: VACUUM (ANALYZE, SKIP LOCKED) parted;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive vac_full_specified commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+WARNING: skipping vacuum of "part1" --- lock not available
+step vac_full_specified: VACUUM (SKIP LOCKED, FULL) part1, part2;
+step commit:
+ COMMIT;
+
+
+starting permutation: lock_access_exclusive vac_full_all_parts commit
+step lock_access_exclusive:
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+
+step vac_full_all_parts: VACUUM (SKIP LOCKED, FULL) parted;
+step commit:
+ COMMIT;
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 99dd7c6bdb..bbb4bd4186 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -72,3 +72,4 @@ test: timeouts
test: vacuum-concurrent-drop
test: predicate-gist
test: predicate-gin
+test: vacuum-skip-locked
diff --git a/src/test/isolation/specs/vacuum-skip-locked.spec b/src/test/isolation/specs/vacuum-skip-locked.spec
new file mode 100644
index 0000000000..3e9e968798
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-skip-locked.spec
@@ -0,0 +1,59 @@
+# Test for the SKIP LOCKED option for VACUUM and ANALYZE commands.
+#
+# This also verifies that log messages are not emitted for skipped relations
+# that were not specified in the VACUUM or ANALYZE command.
+
+setup
+{
+ CREATE TABLE parted (a INT) PARTITION BY LIST (a);
+ CREATE TABLE part1 PARTITION OF parted FOR VALUES IN (1);
+ CREATE TABLE part2 PARTITION OF parted FOR VALUES IN (2);
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS parted;
+}
+
+session "s1"
+step "lock_share"
+{
+ BEGIN;
+ LOCK part1 IN SHARE MODE;
+}
+step "lock_access_exclusive"
+{
+ BEGIN;
+ LOCK part1 IN ACCESS EXCLUSIVE MODE;
+}
+step "commit"
+{
+ COMMIT;
+}
+
+session "s2"
+step "vac_specified" { VACUUM (SKIP LOCKED) part1, part2; }
+step "vac_all_parts" { VACUUM (SKIP LOCKED) parted; }
+step "analyze_specified" { ANALYZE (SKIP LOCKED) part1, part2; }
+step "analyze_all_parts" { ANALYZE (SKIP LOCKED) parted; }
+step "vac_analyze_specified" { VACUUM (ANALYZE, SKIP LOCKED) part1, part2; }
+step "vac_analyze_all_parts" { VACUUM (ANALYZE, SKIP LOCKED) parted; }
+step "vac_full_specified" { VACUUM (SKIP LOCKED, FULL) part1, part2; }
+step "vac_full_all_parts" { VACUUM (SKIP LOCKED, FULL) parted; }
+
+permutation "lock_share" "vac_specified" "commit"
+permutation "lock_share" "vac_all_parts" "commit"
+permutation "lock_share" "analyze_specified" "commit"
+permutation "lock_share" "analyze_all_parts" "commit"
+permutation "lock_share" "vac_analyze_specified" "commit"
+permutation "lock_share" "vac_analyze_all_parts" "commit"
+permutation "lock_share" "vac_full_specified" "commit"
+permutation "lock_share" "vac_full_all_parts" "commit"
+permutation "lock_access_exclusive" "vac_specified" "commit"
+permutation "lock_access_exclusive" "vac_all_parts" "commit"
+permutation "lock_access_exclusive" "analyze_specified" "commit"
+permutation "lock_access_exclusive" "analyze_all_parts" "commit"
+permutation "lock_access_exclusive" "vac_analyze_specified" "commit"
+permutation "lock_access_exclusive" "vac_analyze_all_parts" "commit"
+permutation "lock_access_exclusive" "vac_full_specified" "commit"
+permutation "lock_access_exclusive" "vac_full_all_parts" "commit"
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index d66e2aa3b7..5d4a1e51c0 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -119,6 +119,16 @@ ANALYZE (nonexistant-arg) does_not_exist;
ERROR: syntax error at or near "nonexistant"
LINE 1: ANALYZE (nonexistant-arg) does_not_exist;
^
+-- ensure argument order independence, and that SKIP LOCKED on non-existing
+-- relation still errors out.
+ANALYZE (SKIP LOCKED, VERBOSE) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
+ANALYZE (VERBOSE, SKIP LOCKED) does_not_exist;
+ERROR: relation "does_not_exist" does not exist
+-- SKIP LOCKED option
+VACUUM (SKIP LOCKED) vactst;
+VACUUM (SKIP LOCKED, FULL) vactst;
+ANALYZE (SKIP LOCKED) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 275ce2e270..15749ffd94 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -93,6 +93,16 @@ ANALYZE vactst (i), vacparted (does_not_exist);
ANALYZE (VERBOSE) does_not_exist;
ANALYZE (nonexistant-arg) does_not_exist;
+-- ensure argument order independence, and that SKIP LOCKED on non-existing
+-- relation still errors out.
+ANALYZE (SKIP LOCKED, VERBOSE) does_not_exist;
+ANALYZE (VERBOSE, SKIP LOCKED) does_not_exist;
+
+-- SKIP LOCKED option
+VACUUM (SKIP LOCKED) vactst;
+VACUUM (SKIP LOCKED, FULL) vactst;
+ANALYZE (SKIP LOCKED) vactst;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
--
2.16.2
On Thu, Apr 05, 2018 at 08:29:38PM +0000, Bossart, Nathan wrote:
The new tests are now passing as expected, but I am still doing some
manual testing. Since there is quite a bit of added complexity in
this new patch set, I will certainly not be surprised if this gets
moved to the next commitfest.
That would be wiser. We are two days away from the end of the CF and
this patch gets quite invasive with a set of new concepts, so my
recommendation would be to do so. There has been already much done with
the introduction of the new grammar for ANALYZE, and the introduction of
RangeVarGetRelidExtended().
--
Michael
On Fri, Apr 06, 2018 at 06:41:14AM +0900, Michael Paquier wrote:
That would be wiser. We are two days away from the end of the CF and
this patch gets quite invasive with a set of new concepts, so my
recommendation would be to do so. There has been already much done with
the introduction of the new grammar for ANALYZE, and the introduction of
RangeVarGetRelidExtended().
By the way, let's do that on a new thread next time :)
--
Michael
On 4/5/18, 9:48 PM, "Michael Paquier" <michael@paquier.xyz> wrote:
On Fri, Apr 06, 2018 at 06:41:14AM +0900, Michael Paquier wrote:
That would be wiser. We are two days away from the end of the CF and
this patch gets quite invasive with a set of new concepts, so my
recommendation would be to do so. There has been already much done with
the introduction of the new grammar for ANALYZE, and the introduction of
RangeVarGetRelidExtended().By the way, let's do that on a new thread next time :)
Sounds good. I've bumped it to the next commitfest.
Nathan