improve DEBUG1 logging of parallel workers for CREATE INDEX?

Started by Sami Imseihabout 1 year ago19 messages
#1Sami Imseih
samimseih@gmail.com
1 attachment(s)

While reviewing patch [1]/messages/by-id/CAA5RZ0tS=Fmgku9=qCp-U4EBC-dqVtj+v_EGrKV_NA=THnUDsQ@mail.gmail.com, I realized that the DEBUG1 message
for CREATE INDEX could do better in providing information
about parallel workers launched. Currently, the message just
shows how many workers are planned, but a user may want
to ensure that they have the appropriate number of workers
launched as well when they enable DEBUG1 logging.

Therefore, I want to suggest that instead of:

postgres=# CREATE INDEX tbl_c1 ON tbl(c1);
DEBUG: building index "tbl_c1" on table "tbl" with request for 2
parallel workers
DEBUG: index "tbl_c1" can safely use deduplication
CREATE INDEX

to enhance the message to:

postgres=# create index tbl_c1 on tbl(c1);
DEBUG: building index "tbl_c1" on table "tbl"
DEBUG: launched 0 parallel vacuum workers for index vacuuming (planned: 2)
DEBUG: index "tbl_c1" can safely use deduplication
CREATE INDEX
postgres=#

I prepared a simple patch, attached, for this. The log message matches
the format
used in VACUUM VERBOSE ( for consistency sake ).

Also, I think the documentation in [2]https://www.postgresql.org/docs/current/sql-createindex.html should be updated to mention that DEBUG1
can be used to view parallel worker usage for the build, but I have
not included it in
the patch yet.

Any thoughts?

[1]: /messages/by-id/CAA5RZ0tS=Fmgku9=qCp-U4EBC-dqVtj+v_EGrKV_NA=THnUDsQ@mail.gmail.com

[2]: https://www.postgresql.org/docs/current/sql-createindex.html

Regards,

Sami Imseih
Amazon Web Services (AWS)

Attachments:

0001-improve-DEBUG1-logging-of-parallel-workers-for-CREAT.patchapplication/octet-stream; name=0001-improve-DEBUG1-logging-of-parallel-workers-for-CREAT.patchDownload
From 1cbe4d0aa375df0371da19ca88d2e0dda2708ea2 Mon Sep 17 00:00:00 2001
From: Sami Imseih <simseih@amazon.com>
Date: Fri, 3 Jan 2025 12:16:37 -0600
Subject: [PATCH 1/1] improve DEBUG1 logging of parallel workers for CREATE
 INDEX

---
 src/backend/access/brin/brin.c      | 4 ++++
 src/backend/access/nbtree/nbtsort.c | 4 ++++
 src/backend/catalog/index.c         | 5 ++---
 3 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 9a98454757..f587e13135 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2512,6 +2512,10 @@ _brin_begin_parallel(BrinBuildState *buildstate, Relation heap, Relation index,
 	brinleader->walusage = walusage;
 	brinleader->bufferusage = bufferusage;
 
+	ereport(DEBUG1,
+			(errmsg_internal("launched %d parallel vacuum workers for index vacuuming (planned: %d)",
+							 pcxt->nworkers_launched, pcxt->nworkers_to_launch)));
+
 	/* If no workers were successfully launched, back out (do serial build) */
 	if (pcxt->nworkers_launched == 0)
 	{
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 7aba852db9..b9f39aea73 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -1582,6 +1582,10 @@ _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
 	btleader->walusage = walusage;
 	btleader->bufferusage = bufferusage;
 
+	ereport(DEBUG1,
+			(errmsg_internal("launched %d parallel vacuum workers for index vacuuming (planned: %d)",
+							 pcxt->nworkers_launched, pcxt->nworkers_to_launch)));
+
 	/* If no workers were successfully launched, back out (do serial build) */
 	if (pcxt->nworkers_launched == 0)
 	{
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 221fbb4e28..e7846fd9d4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3007,10 +3007,9 @@ index_build(Relation heapRelation,
 								 RelationGetRelationName(heapRelation))));
 	else
 		ereport(DEBUG1,
-				(errmsg_internal("building index \"%s\" on table \"%s\" with request for %d parallel workers",
+				(errmsg_internal("building index \"%s\" on table \"%s\"",
 								 RelationGetRelationName(indexRelation),
-								 RelationGetRelationName(heapRelation),
-								 indexInfo->ii_ParallelWorkers)));
+								 RelationGetRelationName(heapRelation))));
 
 	/*
 	 * Switch to the table owner's userid, so that any index functions are run
-- 
2.39.5 (Apple Git-154)

#2Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Sami Imseih (#1)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Hi,

On Fri, Jan 03, 2025 at 12:30:14PM -0600, Sami Imseih wrote:

While reviewing patch [1], I realized that the DEBUG1 message
for CREATE INDEX could do better in providing information
about parallel workers launched. Currently, the message just
shows how many workers are planned, but a user may want
to ensure that they have the appropriate number of workers
launched as well when they enable DEBUG1 logging.

Yeah, one could see how many workers are currently running but would not be
able to figure out once the index is created. So adding the info in the log
makes sense to me.

I prepared a simple patch, attached, for this. The log message matches
the format
used in VACUUM VERBOSE ( for consistency sake ).

A few random comments:

=== 1

s/parallel vacuum workers for index vacuuming/parallel workers for index creation/?

(2 times)

=== 2

-  (errmsg_internal("building index \"%s\" on table \"%s\" with request for %d parallel workers",
+  (errmsg_internal("building index \"%s\" on table \"%s\"",

I'd add "in parallel" to match its counterpart "serially" above. That would
make it more clear in case one just look for "building index" in the log.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#3Sami Imseih
samimseih@gmail.com
In reply to: Bertrand Drouvot (#2)
1 attachment(s)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

s/parallel vacuum workers for index vacuuming/parallel workers for index creation/?

oops, that's my oversight from copying the message from vacuum. fixed.

I'd add "in parallel" to match its counterpart "serially" above. That would
make it more clear in case one just look for "building index" in the log.

good point.

Below is the new output with the attached v2.

postgres=# create index foo_idx1 on foo(id);
DEBUG: building index "foo_idx1" on table "foo" with parallel workers
DEBUG: launched 1 parallel workers for index creation (planned: 1)
DEBUG: index "foo_idx1" can safely use deduplication
CREATE INDEX

Regards,

Sami

Attachments:

v2-0001-improve-DEBUG1-logging-of-parallel-workers-for-CR.patchapplication/octet-stream; name=v2-0001-improve-DEBUG1-logging-of-parallel-workers-for-CR.patchDownload
From d011edabb6cda6fce4b06c2b08e389ae11c7470c Mon Sep 17 00:00:00 2001
From: Sami Imseih <simseih@amazon.com>
Date: Wed, 8 Jan 2025 11:20:59 -0600
Subject: [PATCH v2 1/1] improve DEBUG1 logging of parallel workers for CREATE
 INDEX

---
 src/backend/access/brin/brin.c      | 4 ++++
 src/backend/access/nbtree/nbtsort.c | 4 ++++
 src/backend/catalog/index.c         | 5 ++---
 3 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 9a98454757..7f72e5a4f9 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2512,6 +2512,10 @@ _brin_begin_parallel(BrinBuildState *buildstate, Relation heap, Relation index,
 	brinleader->walusage = walusage;
 	brinleader->bufferusage = bufferusage;
 
+	ereport(DEBUG1,
+			(errmsg_internal("launched %d parallel workers for index creation (planned: %d)",
+							 pcxt->nworkers_launched, pcxt->nworkers_to_launch)));
+
 	/* If no workers were successfully launched, back out (do serial build) */
 	if (pcxt->nworkers_launched == 0)
 	{
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 7aba852db9..fe13f5fedf 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -1582,6 +1582,10 @@ _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
 	btleader->walusage = walusage;
 	btleader->bufferusage = bufferusage;
 
+	ereport(DEBUG1,
+			(errmsg_internal("launched %d parallel workers for index creation (planned: %d)",
+							 pcxt->nworkers_launched, pcxt->nworkers_to_launch)));
+
 	/* If no workers were successfully launched, back out (do serial build) */
 	if (pcxt->nworkers_launched == 0)
 	{
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 221fbb4e28..40b97c3885 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3007,10 +3007,9 @@ index_build(Relation heapRelation,
 								 RelationGetRelationName(heapRelation))));
 	else
 		ereport(DEBUG1,
-				(errmsg_internal("building index \"%s\" on table \"%s\" with request for %d parallel workers",
+				(errmsg_internal("building index \"%s\" on table \"%s\" with parallel workers",
 								 RelationGetRelationName(indexRelation),
-								 RelationGetRelationName(heapRelation),
-								 indexInfo->ii_ParallelWorkers)));
+								 RelationGetRelationName(heapRelation))));
 
 	/*
 	 * Switch to the table owner's userid, so that any index functions are run
-- 
2.39.5 (Apple Git-154)

#4Guillaume Lelarge
guillaume@lelarge.info
In reply to: Sami Imseih (#3)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Hi,

Le mer. 8 janv. 2025 à 18:37, Sami Imseih <samimseih@gmail.com> a écrit :

s/parallel vacuum workers for index vacuuming/parallel workers for index

creation/?

oops, that's my oversight from copying the message from vacuum. fixed.

I'd add "in parallel" to match its counterpart "serially" above. That

would

make it more clear in case one just look for "building index" in the log.

good point.

Below is the new output with the attached v2.

postgres=# create index foo_idx1 on foo(id);
DEBUG: building index "foo_idx1" on table "foo" with parallel workers
DEBUG: launched 1 parallel workers for index creation (planned: 1)
DEBUG: index "foo_idx1" can safely use deduplication
CREATE INDEX

I tend to agree it might be better than Benoit's patch on the index
messages, though I'm afraid that DEBUG1 level won't work for many users.
DEBUGx are for developers, not users. A better log level for me would be
LOG. For client messages, LOG messages won't be displayed by default. So
there's still a need for a "SET client_min_messages to LOG" but it won't be
necessary to go all the way to DEBUG1.

Regards.

--
Guillaume.

#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: Guillaume Lelarge (#4)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Guillaume Lelarge <guillaume@lelarge.info> writes:

I tend to agree it might be better than Benoit's patch on the index
messages, though I'm afraid that DEBUG1 level won't work for many users.
DEBUGx are for developers, not users. A better log level for me would be
LOG.

Please, no. That would result in spamming the postmaster log
for perfectly ordinary actions, with no usable way to prevent it.

I think DEBUG is perfectly appropriate for this sort of message.

regards, tom lane

#6Sami Imseih
samimseih@gmail.com
In reply to: Tom Lane (#5)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

I think DEBUG is perfectly appropriate for this sort of message.

I am curious what are the thoughts on introducing a
CREATE INDEX VERBOSE which can provide this info?
similar to users scripting VACUUM VERBOSE to log
more details about the vacuum operation including parallel
usage.

I have not convinced myself this is a good idea, but maybe
someone has an opinion. If not, we can just stick with a better
DEBUG1 message.

Regards,

Sami

#7Guillaume Lelarge
guillaume@lelarge.info
In reply to: Tom Lane (#5)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Le mer. 8 janv. 2025 à 21:35, Tom Lane <tgl@sss.pgh.pa.us> a écrit :

Guillaume Lelarge <guillaume@lelarge.info> writes:

I tend to agree it might be better than Benoit's patch on the index
messages, though I'm afraid that DEBUG1 level won't work for many users.
DEBUGx are for developers, not users. A better log level for me would be
LOG.

Please, no. That would result in spamming the postmaster log
for perfectly ordinary actions, with no usable way to prevent it.

Not sure why I was thinking it would only go to the client. You're right,
it will also go to the logs, and it will be stored by default, which would
be bad. Sorry about this...

I think DEBUG is perfectly appropriate for this sort of message.

Still think DEBUG level is pretty bad for a user experience. Anyway, I much
prefer a DEBUG message than no message at all :)

--
Guillaume.

#8Guillaume Lelarge
guillaume@lelarge.info
In reply to: Sami Imseih (#6)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Le mer. 8 janv. 2025 à 21:44, Sami Imseih <samimseih@gmail.com> a écrit :

I think DEBUG is perfectly appropriate for this sort of message.

I am curious what are the thoughts on introducing a
CREATE INDEX VERBOSE which can provide this info?
similar to users scripting VACUUM VERBOSE to log
more details about the vacuum operation including parallel
usage.

I have not convinced myself this is a good idea, but maybe
someone has an opinion. If not, we can just stick with a better
DEBUG1 message.

Sounds better to me than relying on DEBUG messages. So, yeah, VERBOSE has
my vote.

--
Guillaume.

#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: Sami Imseih (#6)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Sami Imseih <samimseih@gmail.com> writes:

I am curious what are the thoughts on introducing a
CREATE INDEX VERBOSE which can provide this info?
similar to users scripting VACUUM VERBOSE to log
more details about the vacuum operation including parallel
usage.

What I can recall being discussed in the past is to extend
EXPLAIN and/or EXPLAIN ANALYZE to cover utility statements
that have nontrivial execution complexity --- for example,
ALTER TABLE has a lot of machinery underneath, and people
often wish to know things like whether a particular ALTER
will cause a table rewrite or not.

Of course, a patch for that would be a few orders of magnitude
larger than what you've got here :-(. But if you're looking
for a framework for reporting these sorts of details, I'd
much rather go in that direction than follow the model of
VACUUM VERBOSE. VACUUM VERBOSE is a kluge with little to
recommend it other than having been easy to implement.

regards, tom lane

#10Sami Imseih
samimseih@gmail.com
In reply to: Tom Lane (#9)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Of course, a patch for that would be a few orders of magnitude
larger than what you've got here :-(. But if you're looking
for a framework for reporting these sorts of details, I'd
much rather go in that direction than follow the model of
VACUUM VERBOSE. VACUUM VERBOSE is a kluge with little to
recommend it other than having been easy to implement.

To my surprise, REINDEX does have a VERBOSE option.
should have check this earlier :)

postgres=# reindex (verbose) index t_idx1;
INFO: index "t_idx1" was reindexed
DETAIL: CPU: user: 5.33 s, system: 0.48 s, elapsed: 6.26 s
REINDEX

Is there a reason not to do the same for CREATE INDEX?

Also, we can improve the REINDEX verbose message by
also providing the parallel usage.

Regards,

Sami

#11Guillaume Lelarge
guillaume@lelarge.info
In reply to: Sami Imseih (#10)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Le jeu. 9 janv. 2025 à 04:24, Sami Imseih <samimseih@gmail.com> a écrit :

Of course, a patch for that would be a few orders of magnitude
larger than what you've got here :-(. But if you're looking
for a framework for reporting these sorts of details, I'd
much rather go in that direction than follow the model of
VACUUM VERBOSE. VACUUM VERBOSE is a kluge with little to
recommend it other than having been easy to implement.

To my surprise, REINDEX does have a VERBOSE option.
should have check this earlier :)

postgres=# reindex (verbose) index t_idx1;
INFO: index "t_idx1" was reindexed
DETAIL: CPU: user: 5.33 s, system: 0.48 s, elapsed: 6.26 s
REINDEX

Is there a reason not to do the same for CREATE INDEX?

Sounds a good idea to me.

Also, we can improve the REINDEX verbose message by
also providing the parallel usage.

+1

--
Guillaume.

#12Sami Imseih
samimseih@gmail.com
In reply to: Guillaume Lelarge (#11)
1 attachment(s)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

The opinion in this thread is leaning towards a VERBOSE
option and I agree with this as it provides the users with a
much more intuitive way to gather parallel information about
parallel index builds. Also, the current DEBUG1 output is
useless as it is now.

Here is a v1 that implements CREATE INDEX (VERBOSE)
to output the parallel workers planned and launched.

A few points about this patch:

1/ the (VERBOSE) option is added after the INDEX.
It seemed better than adding it after the CREATE keyword
to make the option specific to the "CREATE INDEX" operation
rather than the "CREATE" operation which can be many
things.

2/ The workers launched is determined in the index AM,
therefore a new field called "ii_verbose" is added to IndexInfo
so it can be available to the AM to determine if logging
is required. We currently handle the concurrently
option in the same manner.

3/ currently REINDEX (VERBOSE) does not show parallel
workers planned, but it will with this patch. Both
CREATE INDEX and REINDEX will show the same output.

4/ Updated auto-completed in psql to deal with (VERBOSE)

5/ Removed the existing DEBUG1 output.

6/ Updated the tests for create index/reinde

Here is sample output:

postgres=# CREATE INDEX (VERBOSE ON) ON t(c1);
INFO: building index "t_c1_idx" on table "t" with parallel workers
INFO: launched 1 parallel workers for index creation (planned: 1)
INFO: index "public.t" was created
DETAIL: CPU: user: 0.88 s, system: 0.01 s, elapsed: 0.89 s.
CREATE INDEX
postgres=#
postgres=# REINDEX (verbose) INDEX t_c1_idx ;
INFO: building index "t_c1_idx" on table "t" with parallel workers
INFO: launched 1 parallel workers for index creation (planned: 1)
INFO: index "t_c1_idx" was reindexed
DETAIL: CPU: user: 0.88 s, system: 0.00 s, elapsed: 0.89 s
REINDEX

Looking forward to feedback.

Regards,

Sami

Attachments:

v1-0001-Improve-Parallel-worker-reporting-in-CREATE-INDEX.patchapplication/octet-stream; name=v1-0001-Improve-Parallel-worker-reporting-in-CREATE-INDEX.patchDownload
From bca38498155eba925cd89829781ec0ab33d6cd1f Mon Sep 17 00:00:00 2001
From: Sami Imseih <simseih@amazon.com>
Date: Sat, 18 Jan 2025 16:43:14 -0600
Subject: [PATCH v1 1/1] Improve Parallel worker reporting in CREATE INDEX and
 REINDEX

Prior to this patch, only DEBUG1 logging displayed the number
of parallel workers planned for a CREATE INDEX or REINDEX
operation. Furthermore, the DEBUG1 did not show the number
of workers launched. This patch improves the situation by
allowing for a CREATE INDEX with a VERBOSE option which will
show both the number of parallel workers planned and the number
launched at INFO level. REINDEX already has a VERBOSE option and
it will also show the same parallel information as CREATE INDEX.

In the future, VERBOSE can be expanded to track progress for other
steps being performed by CREATE INDEX or REINDEX.

As parallel workers launched can only be determined in the Index AM's,
a verbose field was added to IndexInfo to instruct the Index AM to print
the logging information.
---
 doc/src/sgml/ref/create_index.sgml         | 23 +++++++++-
 src/backend/access/brin/brin.c             | 11 +++--
 src/backend/access/nbtree/nbtsort.c        | 11 +++--
 src/backend/bootstrap/bootparse.y          |  2 +
 src/backend/catalog/index.c                | 39 +++++++++-------
 src/backend/catalog/toasting.c             |  1 +
 src/backend/commands/indexcmds.c           | 52 +++++++++++++++++++---
 src/backend/commands/tablecmds.c           |  7 +--
 src/backend/nodes/makefuncs.c              |  3 +-
 src/backend/parser/gram.y                  | 52 +++++++++++++---------
 src/backend/tcop/utility.c                 | 21 ++++++++-
 src/bin/psql/tab-complete.in.c             | 14 ++++++
 src/include/catalog/index.h                |  3 +-
 src/include/commands/defrem.h              |  3 +-
 src/include/nodes/execnodes.h              |  3 +-
 src/include/nodes/makefuncs.h              |  3 +-
 src/include/nodes/parsenodes.h             |  1 +
 src/test/regress/expected/create_index.out | 19 ++++++++
 src/test/regress/sql/create_index.sql      | 12 +++++
 19 files changed, 220 insertions(+), 60 deletions(-)

diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/create_index.sgml
index 208389e800..7b6f2eab34 100644
--- a/doc/src/sgml/ref/create_index.sgml
+++ b/doc/src/sgml/ref/create_index.sgml
@@ -21,7 +21,7 @@ PostgreSQL documentation
 
  <refsynopsisdiv>
 <synopsis>
-CREATE [ UNIQUE ] INDEX [ CONCURRENTLY ] [ [ IF NOT EXISTS ] <replaceable class="parameter">name</replaceable> ] ON [ ONLY ] <replaceable class="parameter">table_name</replaceable> [ USING <replaceable class="parameter">method</replaceable> ]
+CREATE [ UNIQUE ] INDEX [ ( VERBOSE <replaceable class="parameter">boolean</replaceable> ) ] [ CONCURRENTLY ] [ [ IF NOT EXISTS ] <replaceable class="parameter">name</replaceable> ] ON [ ONLY ] <replaceable class="parameter">table_name</replaceable> [ USING <replaceable class="parameter">method</replaceable> ]
     ( { <replaceable class="parameter">column_name</replaceable> | ( <replaceable class="parameter">expression</replaceable> ) } [ COLLATE <replaceable class="parameter">collation</replaceable> ] [ <replaceable class="parameter">opclass</replaceable> [ ( <replaceable class="parameter">opclass_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] ) ] ] [ ASC | DESC ] [ NULLS { FIRST | LAST } ] [, ...] )
     [ INCLUDE ( <replaceable class="parameter">column_name</replaceable> [, ...] ) ]
     [ NULLS [ NOT ] DISTINCT ]
@@ -149,6 +149,16 @@ CREATE [ UNIQUE ] INDEX [ CONCURRENTLY ] [ [ IF NOT EXISTS ] <replaceable class=
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><literal>VERBOSE</literal></term>
+      <listitem>
+       <para>
+        Prints information about resource usage during the index creation.
+        at <literal>INFO</literal> level.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><literal>INCLUDE</literal></term>
       <listitem>
@@ -704,6 +714,17 @@ Indexes:
   </refsect2>
  </refsect1>
 
+ <refsect1>
+  <title>Outputs</title>
+
+   <para>
+    When <literal>VERBOSE</literal> is specified, <command>CREATE INDEX</command>
+    reports resource usage by the operation, whether the index is being built
+    serially or with parallel workers, and, for parallel index builds, the number
+    of workers planned and the number of workers launched.
+   </para>
+ </refsect1>
+
  <refsect1>
   <title>Notes</title>
 
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 4289142e20..d6e0aa6a34 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -229,7 +229,7 @@ static void brin_fill_empty_ranges(BrinBuildState *state,
 
 /* parallel index builds */
 static void _brin_begin_parallel(BrinBuildState *buildstate, Relation heap, Relation index,
-								 bool isconcurrent, int request);
+								 bool isconcurrent, int request, bool verbose);
 static void _brin_end_parallel(BrinLeader *brinleader, BrinBuildState *state);
 static Size _brin_parallel_estimate_shared(Relation heap, Snapshot snapshot);
 static double _brin_parallel_heapscan(BrinBuildState *state);
@@ -1161,7 +1161,7 @@ brinbuild(Relation heap, Relation index, IndexInfo *indexInfo)
 	 */
 	if (indexInfo->ii_ParallelWorkers > 0)
 		_brin_begin_parallel(state, heap, index, indexInfo->ii_Concurrent,
-							 indexInfo->ii_ParallelWorkers);
+							 indexInfo->ii_ParallelWorkers, indexInfo->ii_verbose);
 
 	/*
 	 * If parallel build requested and at least one worker process was
@@ -2353,7 +2353,7 @@ check_null_keys(BrinValues *bval, ScanKey *nullkeys, int nnullkeys)
  */
 static void
 _brin_begin_parallel(BrinBuildState *buildstate, Relation heap, Relation index,
-					 bool isconcurrent, int request)
+					 bool isconcurrent, int request, bool verbose)
 {
 	ParallelContext *pcxt;
 	int			scantuplesortstates;
@@ -2512,6 +2512,11 @@ _brin_begin_parallel(BrinBuildState *buildstate, Relation heap, Relation index,
 	brinleader->walusage = walusage;
 	brinleader->bufferusage = bufferusage;
 
+	if (verbose)
+		ereport(INFO,
+				(errmsg_internal("launched %d parallel workers for index creation (planned: %d)",
+								 pcxt->nworkers_launched, pcxt->nworkers_to_launch)));
+
 	/* If no workers were successfully launched, back out (do serial build) */
 	if (pcxt->nworkers_launched == 0)
 	{
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 7aba852db9..f6ce33e17a 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -276,7 +276,7 @@ static void _bt_uppershutdown(BTWriteState *wstate, BTPageState *state);
 static void _bt_load(BTWriteState *wstate,
 					 BTSpool *btspool, BTSpool *btspool2);
 static void _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent,
-							   int request);
+							   int request, bool verbose);
 static void _bt_end_parallel(BTLeader *btleader);
 static Size _bt_parallel_estimate_shared(Relation heap, Snapshot snapshot);
 static double _bt_parallel_heapscan(BTBuildState *buildstate,
@@ -390,7 +390,7 @@ _bt_spools_heapscan(Relation heap, Relation index, BTBuildState *buildstate,
 	/* Attempt to launch parallel worker scan when required */
 	if (indexInfo->ii_ParallelWorkers > 0)
 		_bt_begin_parallel(buildstate, indexInfo->ii_Concurrent,
-						   indexInfo->ii_ParallelWorkers);
+						   indexInfo->ii_ParallelWorkers, indexInfo->ii_verbose);
 
 	/*
 	 * If parallel build requested and at least one worker process was
@@ -1395,7 +1395,7 @@ _bt_load(BTWriteState *wstate, BTSpool *btspool, BTSpool *btspool2)
  * never set, and caller should proceed with a serial index build.
  */
 static void
-_bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
+_bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request, bool verbose)
 {
 	ParallelContext *pcxt;
 	int			scantuplesortstates;
@@ -1582,6 +1582,11 @@ _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
 	btleader->walusage = walusage;
 	btleader->bufferusage = bufferusage;
 
+	if (verbose)
+		ereport(INFO,
+				(errmsg_internal("launched %d parallel workers for index creation (planned: %d)",
+								 pcxt->nworkers_launched, pcxt->nworkers_to_launch)));
+
 	/* If no workers were successfully launched, back out (do serial build) */
 	if (pcxt->nworkers_launched == 0)
 	{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 9833f52c1b..0583bb3bcb 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -318,6 +318,7 @@ Boot_DeclareIndexStmt:
 								false,
 								false,
 								true, /* skip_build */
+								false,
 								false);
 					do_end();
 				}
@@ -371,6 +372,7 @@ Boot_DeclareUniqueIndexStmt:
 								false,
 								false,
 								true, /* skip_build */
+								false,
 								false);
 					do_end();
 				}
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7377912b41..2952b4f9e1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1396,7 +1396,8 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId,
 							false,	/* not ready for inserts */
 							true,
 							indexRelation->rd_indam->amsummarizing,
-							oldInfo->ii_WithoutOverlaps);
+							oldInfo->ii_WithoutOverlaps,
+							false);
 
 	/*
 	 * Extract the list of column names and the column numbers for the new
@@ -1482,7 +1483,8 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId,
  */
 void
 index_concurrently_build(Oid heapRelationId,
-						 Oid indexRelationId)
+						 Oid indexRelationId,
+						 bool verbose)
 {
 	Relation	heapRel;
 	Oid			save_userid;
@@ -1519,6 +1521,7 @@ index_concurrently_build(Oid heapRelationId,
 	Assert(!indexInfo->ii_ReadyForInserts);
 	indexInfo->ii_Concurrent = true;
 	indexInfo->ii_BrokenHotChain = false;
+	indexInfo->ii_verbose = verbose;
 
 	/* Now build the index */
 	index_build(heapRel, indexRelation, indexInfo, false, true);
@@ -2451,7 +2454,8 @@ BuildIndexInfo(Relation index)
 					   indexStruct->indisready,
 					   false,
 					   index->rd_indam->amsummarizing,
-					   indexStruct->indisexclusion && indexStruct->indisunique);
+					   indexStruct->indisexclusion && indexStruct->indisunique,
+					   false);
 
 	/* fill in attribute numbers */
 	for (i = 0; i < numAtts; i++)
@@ -2511,7 +2515,8 @@ BuildDummyIndexInfo(Relation index)
 					   indexStruct->indisready,
 					   false,
 					   index->rd_indam->amsummarizing,
-					   indexStruct->indisexclusion && indexStruct->indisunique);
+					   indexStruct->indisexclusion && indexStruct->indisunique,
+					   false);
 
 	/* fill in attribute numbers */
 	for (i = 0; i < numAtts; i++)
@@ -3001,17 +3006,19 @@ index_build(Relation heapRelation,
 			plan_create_index_workers(RelationGetRelid(heapRelation),
 									  RelationGetRelid(indexRelation));
 
-	if (indexInfo->ii_ParallelWorkers == 0)
-		ereport(DEBUG1,
-				(errmsg_internal("building index \"%s\" on table \"%s\" serially",
-								 RelationGetRelationName(indexRelation),
-								 RelationGetRelationName(heapRelation))));
-	else
-		ereport(DEBUG1,
-				(errmsg_internal("building index \"%s\" on table \"%s\" with request for %d parallel workers",
-								 RelationGetRelationName(indexRelation),
-								 RelationGetRelationName(heapRelation),
-								 indexInfo->ii_ParallelWorkers)));
+	if (indexInfo->ii_verbose)
+	{
+		if (indexInfo->ii_ParallelWorkers == 0)
+			ereport(INFO,
+					(errmsg_internal("building index \"%s\" on table \"%s\" serially",
+									 RelationGetRelationName(indexRelation),
+									 RelationGetRelationName(heapRelation))));
+		else
+			ereport(INFO,
+					(errmsg_internal("building index \"%s\" on table \"%s\" with parallel workers",
+									 RelationGetRelationName(indexRelation),
+									 RelationGetRelationName(heapRelation))));
+	}
 
 	/*
 	 * Switch to the table owner's userid, so that any index functions are run
@@ -3774,6 +3781,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
 		indexInfo->ii_ExclusionStrats = NULL;
 	}
 
+	indexInfo->ii_verbose = params->options & REINDEXOPT_VERBOSE;
+
 	/* Suppress use of the target index while rebuilding it */
 	SetReindexProcessing(heapId, indexId);
 
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 874a8fc89a..57f9e9cbce 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -308,6 +308,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
 	indexInfo->ii_Am = BTREE_AM_OID;
 	indexInfo->ii_AmCache = NULL;
 	indexInfo->ii_Context = CurrentMemoryContext;
+	indexInfo->ii_verbose = false;
 
 	collationIds[0] = InvalidOid;
 	collationIds[1] = InvalidOid;
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 59c836fc24..1257becea5 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -244,7 +244,8 @@ CheckIndexCompatible(Oid oldId,
 	 */
 	indexInfo = makeIndexInfo(numberOfAttributes, numberOfAttributes,
 							  accessMethodId, NIL, NIL, false, false,
-							  false, false, amsummarizing, isWithoutOverlaps);
+							  false, false, amsummarizing, isWithoutOverlaps,
+							  false);
 	typeIds = palloc_array(Oid, numberOfAttributes);
 	collationIds = palloc_array(Oid, numberOfAttributes);
 	opclassIds = palloc_array(Oid, numberOfAttributes);
@@ -550,7 +551,8 @@ DefineIndex(Oid tableId,
 			bool check_rights,
 			bool check_not_in_use,
 			bool skip_build,
-			bool quiet)
+			bool quiet,
+			bool verbose)
 {
 	bool		concurrent;
 	char	   *indexRelationName;
@@ -591,11 +593,14 @@ DefineIndex(Oid tableId,
 	Oid			root_save_userid;
 	int			root_save_sec_context;
 	int			root_save_nestlevel;
+	PGRUsage	ru0;
 
 	root_save_nestlevel = NewGUCNestLevel();
 
 	RestrictSearchPath();
 
+	pg_rusage_init(&ru0);
+
 	/*
 	 * Some callers need us to run with an empty default_tablespace; this is a
 	 * necessary hack to be able to reproduce catalog state accurately when
@@ -929,7 +934,8 @@ DefineIndex(Oid tableId,
 							  !concurrent,
 							  concurrent,
 							  amissummarizing,
-							  stmt->iswithoutoverlaps);
+							  stmt->iswithoutoverlaps,
+							  verbose);
 
 	typeIds = palloc_array(Oid, numberOfAttributes);
 	collationIds = palloc_array(Oid, numberOfAttributes);
@@ -1246,7 +1252,16 @@ DefineIndex(Oid tableId,
 
 		/* If this is the top-level index, we're done */
 		if (!OidIsValid(parentIndexId))
+		{
+			/* Log what we did */
+			if (indexInfo->ii_verbose)
+				ereport(INFO,
+						(errmsg("index \"%s.%s\" was created",
+								get_namespace_name(RelationGetNamespace(rel)), RelationGetRelationName(rel)),
+						 errdetail("%s.", pg_rusage_show(&ru0))));
+
 			pgstat_progress_end_command();
+		}
 
 		return address;
 	}
@@ -1502,7 +1517,7 @@ DefineIndex(Oid tableId,
 									-1,
 									is_alter_table, check_rights,
 									check_not_in_use,
-									skip_build, quiet);
+									skip_build, quiet, false);
 					SetUserIdAndSecContext(child_save_userid,
 										   child_save_sec_context);
 
@@ -1559,7 +1574,16 @@ DefineIndex(Oid tableId,
 		SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
 		table_close(rel, NoLock);
 		if (!OidIsValid(parentIndexId))
+		{
+			/* Log what we did */
+			if (indexInfo->ii_verbose)
+				ereport(INFO,
+						(errmsg("index \"%s.%s\" was created",
+								get_namespace_name(RelationGetNamespace(rel)), RelationGetRelationName(rel)),
+						 errdetail("%s.", pg_rusage_show(&ru0))));
+
 			pgstat_progress_end_command();
+		}
 		else
 		{
 			/* Update progress for an intermediate partitioned index itself */
@@ -1582,7 +1606,16 @@ DefineIndex(Oid tableId,
 		 * otherwise, increment progress to report one child index is done.
 		 */
 		if (!OidIsValid(parentIndexId))
+		{
+			/* Log what we did */
+			if (indexInfo->ii_verbose)
+				ereport(INFO,
+						(errmsg("index \"%s.%s\" was created",
+								get_namespace_name(RelationGetNamespace(rel)), RelationGetRelationName(rel)),
+						 errdetail("%s.", pg_rusage_show(&ru0))));
+
 			pgstat_progress_end_command();
+		}
 		else
 			pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
 
@@ -1682,7 +1715,7 @@ DefineIndex(Oid tableId,
 	PushActiveSnapshot(GetTransactionSnapshot());
 
 	/* Perform concurrent build of index */
-	index_concurrently_build(tableId, indexRelationId);
+	index_concurrently_build(tableId, indexRelationId, verbose);
 
 	/* we can do away with our snapshot */
 	PopActiveSnapshot();
@@ -1798,6 +1831,13 @@ DefineIndex(Oid tableId,
 	 */
 	UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
 
+	/* Log what we did */
+	if (indexInfo->ii_verbose)
+		ereport(INFO,
+				(errmsg("index \"%s.%s\" was created",
+						get_namespace_name(RelationGetNamespace(rel)), RelationGetRelationName(rel)),
+				 errdetail("%s.", pg_rusage_show(&ru0))));
+
 	pgstat_progress_end_command();
 
 	return address;
@@ -4096,7 +4136,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
 		pgstat_progress_update_multi_param(4, progress_index, progress_vals);
 
 		/* Perform concurrent build of new index */
-		index_concurrently_build(newidx->tableId, newidx->indexId);
+		index_concurrently_build(newidx->tableId, newidx->indexId, params->options & REINDEXOPT_VERBOSE);
 
 		PopActiveSnapshot();
 		CommitTransactionCommand();
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d2420a9558..7b04656538 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1275,7 +1275,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
 						RelationGetRelid(idxRel),
 						constraintOid,
 						-1,
-						false, false, false, false, false);
+						false, false, false, false, false, false);
 
 			index_close(idxRel, AccessShareLock);
 		}
@@ -9320,7 +9320,8 @@ ATExecAddIndex(AlteredTableInfo *tab, Relation rel,
 						  check_rights,
 						  false,	/* check_not_in_use - we did it already */
 						  skip_build,
-						  quiet);
+						  quiet,
+						  false);
 
 	/*
 	 * If TryReuseIndex() stashed a relfilenumber for us, we used it for the
@@ -19624,7 +19625,7 @@ AttachPartitionEnsureIndexes(List **wqueue, Relation rel, Relation attachrel)
 						RelationGetRelid(idxRel),
 						conOid,
 						-1,
-						true, false, false, false, false);
+						true, false, false, false, false, false);
 		}
 
 		index_close(idxRel, AccessShareLock);
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 007612563c..60e3a67fa7 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -787,7 +787,7 @@ IndexInfo *
 makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
 			  List *predicates, bool unique, bool nulls_not_distinct,
 			  bool isready, bool concurrent, bool summarizing,
-			  bool withoutoverlaps)
+			  bool withoutoverlaps, bool verbose)
 {
 	IndexInfo  *n = makeNode(IndexInfo);
 
@@ -803,6 +803,7 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
 	n->ii_Concurrent = concurrent;
 	n->ii_Summarizing = summarizing;
 	n->ii_WithoutOverlaps = withoutoverlaps;
+	n->ii_verbose = verbose;
 
 	/* summarizing indexes cannot contain non-key attributes */
 	Assert(!summarizing || (numkeyattrs == numattrs));
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d7f9c00c40..349db1854e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -559,6 +559,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 %type <ival>	reindex_target_relation reindex_target_all
 %type <list>	opt_reindex_option_list
 
+%type <list>	opt_index_option_list
+
 %type <node>	copy_generic_opt_arg copy_generic_opt_arg_list_item
 %type <defelt>	copy_generic_opt_elem
 %type <list>	copy_generic_opt_list copy_generic_opt_arg_list
@@ -8158,23 +8160,24 @@ defacl_privilege_target:
  * willing to make TABLESPACE a fully reserved word.
  *****************************************************************************/
 
-IndexStmt:	CREATE opt_unique INDEX opt_concurrently opt_single_name
+IndexStmt:	CREATE opt_unique INDEX opt_index_option_list opt_concurrently opt_single_name
 			ON relation_expr access_method_clause '(' index_params ')'
 			opt_include opt_unique_null_treatment opt_reloptions OptTableSpace where_clause
 				{
 					IndexStmt *n = makeNode(IndexStmt);
 
 					n->unique = $2;
-					n->concurrent = $4;
-					n->idxname = $5;
-					n->relation = $7;
-					n->accessMethod = $8;
-					n->indexParams = $10;
-					n->indexIncludingParams = $12;
-					n->nulls_not_distinct = !$13;
-					n->options = $14;
-					n->tableSpace = $15;
-					n->whereClause = $16;
+					n->params = $4;
+					n->concurrent = $5;
+					n->idxname = $6;
+					n->relation = $8;
+					n->accessMethod = $9;
+					n->indexParams = $11;
+					n->indexIncludingParams = $13;
+					n->nulls_not_distinct = !$14;
+					n->options = $15;
+					n->tableSpace = $16;
+					n->whereClause = $17;
 					n->excludeOpNames = NIL;
 					n->idxcomment = NULL;
 					n->indexOid = InvalidOid;
@@ -8190,23 +8193,24 @@ IndexStmt:	CREATE opt_unique INDEX opt_concurrently opt_single_name
 					n->reset_default_tblspc = false;
 					$$ = (Node *) n;
 				}
-			| CREATE opt_unique INDEX opt_concurrently IF_P NOT EXISTS name
+			| CREATE opt_unique INDEX opt_index_option_list opt_concurrently IF_P NOT EXISTS name
 			ON relation_expr access_method_clause '(' index_params ')'
 			opt_include opt_unique_null_treatment opt_reloptions OptTableSpace where_clause
 				{
 					IndexStmt *n = makeNode(IndexStmt);
 
 					n->unique = $2;
-					n->concurrent = $4;
-					n->idxname = $8;
-					n->relation = $10;
-					n->accessMethod = $11;
-					n->indexParams = $13;
-					n->indexIncludingParams = $15;
-					n->nulls_not_distinct = !$16;
-					n->options = $17;
-					n->tableSpace = $18;
-					n->whereClause = $19;
+					n->params = $4;
+					n->concurrent = $5;
+					n->idxname = $9;
+					n->relation = $11;
+					n->accessMethod = $12;
+					n->indexParams = $14;
+					n->indexIncludingParams = $16;
+					n->nulls_not_distinct = !$17;
+					n->options = $18;
+					n->tableSpace = $19;
+					n->whereClause = $20;
 					n->excludeOpNames = NIL;
 					n->idxcomment = NULL;
 					n->indexOid = InvalidOid;
@@ -8238,6 +8242,10 @@ index_params:	index_elem							{ $$ = list_make1($1); }
 			| index_params ',' index_elem			{ $$ = lappend($1, $3); }
 		;
 
+opt_index_option_list:
+			'(' utility_option_list ')'				{ $$ = $2; }
+			| /* EMPTY */							{ $$ = NULL; }
+		;
 
 index_elem_options:
 	opt_collate opt_qualified_name opt_asc_desc opt_nulls_order
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 25fe3d5801..c44e0b9ad7 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1459,11 +1459,28 @@ ProcessUtilitySlow(ParseState *pstate,
 					LOCKMODE	lockmode;
 					int			nparts = -1;
 					bool		is_alter_table;
+					bool		verbose = false;
+					ListCell *lc;
 
 					if (stmt->concurrent)
 						PreventInTransactionBlock(isTopLevel,
 												  "CREATE INDEX CONCURRENTLY");
 
+					/* Parse option list */
+					foreach(lc, stmt->params)
+					{
+						DefElem    *opt = (DefElem *) lfirst(lc);
+
+						if (strcmp(opt->defname, "verbose") == 0)
+							verbose = defGetBoolean(opt);
+						else
+							ereport(ERROR,
+								(errcode(ERRCODE_SYNTAX_ERROR),
+								 errmsg("unrecognized CREATE INDEX option \"%s\"",
+								 opt->defname),
+								 parser_errposition(pstate, opt->location)));
+					}
+
 					/*
 					 * Look up the relation OID just once, right here at the
 					 * beginning, so that we don't end up repeating the name
@@ -1495,7 +1512,6 @@ ProcessUtilitySlow(ParseState *pstate,
 					if (stmt->relation->inh &&
 						get_rel_relkind(relid) == RELKIND_PARTITIONED_TABLE)
 					{
-						ListCell   *lc;
 						List	   *inheritors = NIL;
 
 						inheritors = find_all_inheritors(relid, lockmode, NULL);
@@ -1552,7 +1568,8 @@ ProcessUtilitySlow(ParseState *pstate,
 									true,	/* check_rights */
 									true,	/* check_not_in_use */
 									false,	/* skip_build */
-									false); /* quiet */
+									false,  /* quiet */
+									verbose); /* verbose */
 
 					/*
 					 * Add the CREATE INDEX node itself to stash right away;
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 81cbf10aa2..05ad4b8110 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -3373,6 +3373,20 @@ match_previous_words(int pattern_id,
 	else if (TailMatches("CREATE|UNIQUE", "INDEX", "CONCURRENTLY"))
 		COMPLETE_WITH_SCHEMA_QUERY_PLUS(Query_for_list_of_indexes,
 										"ON");
+
+	/* Complete CREATE|UNIQUE INDEX with "VERBOSE" */
+	else if (TailMatches("CREATE|UNIQUE", "INDEX", "(*") &&
+			 !TailMatches("CREATE|UNIQUE", "INDEX", "(*)"))
+	{
+		/*
+		 * This fires if we're in an unfinished parenthesized option list.
+		 * get_previous_words treats a completed parenthesized option list as
+		 * one word, so the above test is correct.
+		 */
+		if (ends_with(prev_wd, '('))
+			COMPLETE_WITH( "VERBOSE");
+	}
+
 	/* Complete CREATE|UNIQUE INDEX [CONCURRENTLY] <sth> with "ON" */
 	else if (TailMatches("CREATE|UNIQUE", "INDEX", MatchAny) ||
 			 TailMatches("CREATE|UNIQUE", "INDEX", "CONCURRENTLY", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 4daa8bef5e..d82026d7a4 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -101,7 +101,8 @@ extern Oid	index_concurrently_create_copy(Relation heapRelation,
 										   const char *newName);
 
 extern void index_concurrently_build(Oid heapRelationId,
-									 Oid indexRelationId);
+									 Oid indexRelationId,
+									 bool verbose);
 
 extern void index_concurrently_swap(Oid newIndexId,
 									Oid oldIndexId,
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 6d9348bac8..1776e76250 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -35,7 +35,8 @@ extern ObjectAddress DefineIndex(Oid tableId,
 								 bool check_rights,
 								 bool check_not_in_use,
 								 bool skip_build,
-								 bool quiet);
+								 bool quiet,
+								 bool verbose);
 extern void ExecReindex(ParseState *pstate, const ReindexStmt *stmt, bool isTopLevel);
 extern char *makeObjectName(const char *name1, const char *name2,
 							const char *label);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index d0f2dca592..db75dbb2d6 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -185,7 +185,7 @@ typedef struct ExprState
  *		AmCache				private cache area for index AM
  *		Context				memory context holding this IndexInfo
  *
- * ii_Concurrent, ii_BrokenHotChain, and ii_ParallelWorkers are used only
+ * ii_Concurrent, ii_BrokenHotChain, and ii_ParallelWorkers, ii_verbose are used only
  * during index build; they're conventionally zeroed otherwise.
  * ----------------
  */
@@ -218,6 +218,7 @@ typedef struct IndexInfo
 	Oid			ii_Am;
 	void	   *ii_AmCache;
 	MemoryContext ii_Context;
+	bool		ii_verbose;
 } IndexInfo;
 
 /* ----------------
diff --git a/src/include/nodes/makefuncs.h b/src/include/nodes/makefuncs.h
index 5473ce9a28..288cbe5035 100644
--- a/src/include/nodes/makefuncs.h
+++ b/src/include/nodes/makefuncs.h
@@ -99,7 +99,8 @@ extern IndexInfo *makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid,
 								List *expressions, List *predicates,
 								bool unique, bool nulls_not_distinct,
 								bool isready, bool concurrent,
-								bool summarizing, bool withoutoverlaps);
+								bool summarizing, bool withoutoverlaps,
+								bool verbose);
 
 extern Node *makeStringConst(char *str, int location);
 extern DefElem *makeDefElem(char *name, Node *arg, int location);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ffe155ee20..60d78acdad 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3449,6 +3449,7 @@ typedef struct IndexStmt
 	bool		if_not_exists;	/* just do nothing if index already exists? */
 	bool		reset_default_tblspc;	/* reset default_tablespace prior to
 										 * executing */
+	List		*params;		/* list of DefElem nodes */
 } IndexStmt;
 
 /* ----------------------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 8011c141bf..c1b4052280 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -1449,6 +1449,19 @@ DROP FUNCTION predicate_stable();
 BEGIN;
 CREATE INDEX std_index on concur_heap(f2);
 COMMIT;
+-- CREATE INDEX (VERBOSE)
+CREATE TABLE verbose_test (id int);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+CREATE INDEX (VERBOSE) verbose_test_id ON verbose_test(id);
+INFO:  building index "verbose_test_id" on table "verbose_test" serially
+INFO:  index "public.verbose_test" was created
+ALTER TABLE verbose_test SET (parallel_workers = 1);
+CREATE INDEX (VERBOSE) verbose_test_id2 ON verbose_test(id);
+INFO:  building index "verbose_test_id2" on table "verbose_test" with parallel workers
+INFO:  launched 1 parallel workers for index creation (planned: 1)
+INFO:  index "public.verbose_test" was created
+\set VERBOSITY default
+DROP TABLE  verbose_test;
 -- Failed builds are left invalid by VACUUM FULL, fixed by REINDEX
 VACUUM FULL concur_heap;
 REINDEX TABLE concur_heap;
@@ -2515,6 +2528,12 @@ explain (costs off)
 CREATE TABLE reindex_verbose(id integer primary key);
 \set VERBOSITY terse \\ -- suppress machine-dependent details
 REINDEX (VERBOSE) TABLE reindex_verbose;
+INFO:  building index "reindex_verbose_pkey" on table "reindex_verbose" serially
+INFO:  index "reindex_verbose_pkey" was reindexed
+ALTER TABLE reindex_verbose SET (parallel_workers = 1);
+REINDEX (VERBOSE) TABLE reindex_verbose;
+INFO:  building index "reindex_verbose_pkey" on table "reindex_verbose" with parallel workers
+INFO:  launched 1 parallel workers for index creation (planned: 1)
 INFO:  index "reindex_verbose_pkey" was reindexed
 \set VERBOSITY default
 DROP TABLE reindex_verbose;
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 068c66b95a..fe7fc97f79 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -526,6 +526,16 @@ BEGIN;
 CREATE INDEX std_index on concur_heap(f2);
 COMMIT;
 
+-- CREATE INDEX (VERBOSE)
+CREATE TABLE verbose_test (id int);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+CREATE INDEX (VERBOSE) verbose_test_id ON verbose_test(id);
+ALTER TABLE verbose_test SET (parallel_workers = 1);
+CREATE INDEX (VERBOSE) verbose_test_id2 ON verbose_test(id);
+\set VERBOSITY default
+DROP TABLE  verbose_test;
+
+
 -- Failed builds are left invalid by VACUUM FULL, fixed by REINDEX
 VACUUM FULL concur_heap;
 REINDEX TABLE concur_heap;
@@ -968,6 +978,8 @@ explain (costs off)
 CREATE TABLE reindex_verbose(id integer primary key);
 \set VERBOSITY terse \\ -- suppress machine-dependent details
 REINDEX (VERBOSE) TABLE reindex_verbose;
+ALTER TABLE reindex_verbose SET (parallel_workers = 1);
+REINDEX (VERBOSE) TABLE reindex_verbose;
 \set VERBOSITY default
 DROP TABLE reindex_verbose;
 
-- 
2.39.5 (Apple Git-154)

#13Michael Paquier
michael@paquier.xyz
In reply to: Sami Imseih (#12)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

On Sat, Jan 18, 2025 at 05:42:50PM -0600, Sami Imseih wrote:

Here is a v1 that implements CREATE INDEX (VERBOSE)
to output the parallel workers planned and launched.

Hmm. I am reading Tom's opinion that goes toward not going in this
direction for more commands, with the point to extend EXPLAIN to show
this kind of information:
/messages/by-id/1692530.1736369905@sss.pgh.pa.us

So do we really want to do what's proposed here? I'm +-0 about the
VERBOSE option attached to more commands, as it is a bit harder for
clients to catch the information wanted. So here comes my question:
how do we want to consume this information at the end from the
perspective of the client?
--
Michael

#14Sami Imseih
samimseih@gmail.com
In reply to: Michael Paquier (#13)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Hmm. I am reading Tom's opinion that goes toward not going in this
direction for more commands, with the point to extend EXPLAIN to show
this kind of information:
/messages/by-id/1692530.1736369905@sss.pgh.pa.us

That sounds like the ability to do something like EXPLAIN CREATE INDEX ...
is that correct?

So do we really want to do what's proposed here? I'm +-0 about the
VERBOSE option attached to more commands, as it is a bit harder for
clients to catch the information wanted. So here comes my question:
how do we want to consume this information at the end from the
perspective of the client?

For interactive usage in psql or pgadmin, it's trivial to capture this
information. The
information can also be written to the server log with log_min_messages=INFO

A bit more work is required to redirect messages to an out file from psql, as
you need to ensure that stderr is redirected to a file.
It's also a bit more work to capture this information from something like a
JDBC application.

IMO the interactive use-case is where this is the most useful as you can start a
CREATE INDEX (VERBOSE) and ensure that it's going to launch all the parallel
workers that it planned before letting it it continue; or control-c and figure
out why not all planned workers launched.

Regards,

Sami

#15Guillaume Lelarge
guillaume@lelarge.info
In reply to: Sami Imseih (#14)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Hi,

Le dim. 19 janv. 2025 à 06:40, Sami Imseih <samimseih@gmail.com> a écrit :

Hmm. I am reading Tom's opinion that goes toward not going in this
direction for more commands, with the point to extend EXPLAIN to show
this kind of information:
/messages/by-id/1692530.1736369905@sss.pgh.pa.us

That sounds like the ability to do something like EXPLAIN CREATE INDEX ...
is that correct?

Yes, and I suppose this would be quite some work to do. Though I see how
interesting it would be, and a nice project to hack on.

So do we really want to do what's proposed here? I'm +-0 about the
VERBOSE option attached to more commands, as it is a bit harder for
clients to catch the information wanted.

It sounds to me a lot easier to know about a VERBOSE option that gives you
more information, than using a DEBUG level to get the same information.
DEBUG level messages aren't explained in the CREATE INDEX manpage, whereas
a VERBOSE option would be.

So here comes my question:

how do we want to consume this information at the end from the
perspective of the client?

For interactive usage in psql or pgadmin, it's trivial to capture this
information.

+1

The information can also be written to the server log with

log_min_messages=INFO

A bit more work is required to redirect messages to an out file from psql,
as
you need to ensure that stderr is redirected to a file.
It's also a bit more work to capture this information from something like a
JDBC application.

IMO the interactive use-case is where this is the most useful as you can
start a
CREATE INDEX (VERBOSE) and ensure that it's going to launch all the
parallel
workers that it planned before letting it it continue; or control-c and
figure
out why not all planned workers launched.

--
Guillaume.

#16Sami Imseih
samimseih@gmail.com
In reply to: Guillaume Lelarge (#15)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Hmm. I am reading Tom's opinion that goes toward not going in this
direction for more commands, with the point to extend EXPLAIN to show
this kind of information:
/messages/by-id/1692530.1736369905@sss.pgh.pa.us

That sounds like the ability to do something like EXPLAIN CREATE INDEX ...
is that correct?

Yes, and I suppose this would be quite some work to do. Though I see how
interesting it would be, and a nice project to hack on.

I have been contemplating this and do see how something like EXPLAIN
could be useful, as it would give users the ability to examine
what a command will do, i.e. EXPLAIN CREATE INDEX will tell you
how many parallel workers are planned and EXPLAIN (ANALYZE) CREATE
INDEX will actually execute the command and provide the actual parallel
workers launched. Tom also mentioned the example of a user wanting to know if
a table rewrite will occur. There are other cases where this could be useful.

FWIW, Oracle does support EXPLAIN PLAN FOR CREATE INDEX and
EXPLAIN PLAN FOR REBUILD (for index rebuilds) [1]https://oracle-randolf.blogspot.com/2009/02/explain-plan-on-ddls.html[2]https://stackoverflow.com/questions/827123/how-can-i-estimate-the-size-of-an-oracle-index, but this functionality
is not officially documented [3]https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/EXPLAIN-PLAN.html. Oracle also provides a documented function to
estimate the size of an index [4]https://docs.oracle.com/en/database/oracle/oracle-database/21/arpls/DBMS_SPACE.html#GUID-B6EB7527-BC67-4394-9E7A-01F2790C409A. I do not have access to an Oracle instance to
verify the state of current versions of Oracle, but I do vaguely remember this
from when I worked on Oracle many years ago :)

Now, I also realized that v1 does not include logging for ALTER TABLE commands
that add an index, such as "ALTER TABLE foo ADD CONSTRAINT foo_uq UNIQUE (c1)"
or attaching a partition and ensuring the index is built. So this
needs to be added to
the current proposal.

So far these are 2 options being discussed:

1/
The current proposal of:

CREATE INDEX (VERBOSE) and ALTER TABLE (VERBOSE)
and output debugging at INFO level.

2/
EXPLAIN

This is a lot more work. Currently QueryDesc has all the info we
need to generate the execution plan. But, we will likely need a
new struct that can track the debugging info to relay it back
up to explain. And this has to be generic enough for all cases.

Also, implementing an EXPLAIN and EXPLAIN ANALYZE for such commands
may not even be feasible. Plans are different in that they are generated
up-front and we can choose to execute them or not. This is not the same
for the other types of commands being discussed.

I am not convinced it's worth the effort.

thoughts? other approaches?

[1]: https://oracle-randolf.blogspot.com/2009/02/explain-plan-on-ddls.html
[2]: https://stackoverflow.com/questions/827123/how-can-i-estimate-the-size-of-an-oracle-index
[3]: https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/EXPLAIN-PLAN.html
[4]: https://docs.oracle.com/en/database/oracle/oracle-database/21/arpls/DBMS_SPACE.html#GUID-B6EB7527-BC67-4394-9E7A-01F2790C409A

Regards,

Sami

#17Guillaume Lelarge
guillaume@lelarge.info
In reply to: Sami Imseih (#16)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

Le lun. 20 janv. 2025 à 04:28, Sami Imseih <samimseih@gmail.com> a écrit :

Hmm. I am reading Tom's opinion that goes toward not going in this
direction for more commands, with the point to extend EXPLAIN to show
this kind of information:

/messages/by-id/1692530.1736369905@sss.pgh.pa.us

That sounds like the ability to do something like EXPLAIN CREATE INDEX

...

is that correct?

Yes, and I suppose this would be quite some work to do. Though I see how
interesting it would be, and a nice project to hack on.

I have been contemplating this and do see how something like EXPLAIN
could be useful, as it would give users the ability to examine
what a command will do, i.e. EXPLAIN CREATE INDEX will tell you
how many parallel workers are planned and EXPLAIN (ANALYZE) CREATE
INDEX will actually execute the command and provide the actual parallel
workers launched. Tom also mentioned the example of a user wanting to know
if
a table rewrite will occur. There are other cases where this could be
useful.

FWIW, Oracle does support EXPLAIN PLAN FOR CREATE INDEX and
EXPLAIN PLAN FOR REBUILD (for index rebuilds) [1][2], but this
functionality
is not officially documented [3]. Oracle also provides a documented
function to
estimate the size of an index [4]. I do not have access to an Oracle
instance to
verify the state of current versions of Oracle, but I do vaguely remember
this
from when I worked on Oracle many years ago :)

Now, I also realized that v1 does not include logging for ALTER TABLE
commands
that add an index, such as "ALTER TABLE foo ADD CONSTRAINT foo_uq UNIQUE
(c1)"
or attaching a partition and ensuring the index is built. So this
needs to be added to
the current proposal.

So far these are 2 options being discussed:

1/
The current proposal of:

CREATE INDEX (VERBOSE) and ALTER TABLE (VERBOSE)
and output debugging at INFO level.

I guess the major issue is that it won't scale fine.

2/
EXPLAIN

This is a lot more work. Currently QueryDesc has all the info we
need to generate the execution plan. But, we will likely need a
new struct that can track the debugging info to relay it back
up to explain. And this has to be generic enough for all cases.

Also, implementing an EXPLAIN and EXPLAIN ANALYZE for such commands
may not even be feasible. Plans are different in that they are generated
up-front and we can choose to execute them or not. This is not the same
for the other types of commands being discussed.

I am not convinced it's worth the effort.

thoughts? other approaches?

You might be interested by this thread "Thinking about EXPLAIN ALTER TABLE":

/messages/by-id/CAM-w4HNm1M5J-ow8UjTcqRe3JPxkVCrGe56tRpPUSePSdGcZ_w@mail.gmail.com

I didn't had the time to read the whole thread, but it might be quite
interesting.

[1] https://oracle-randolf.blogspot.com/2009/02/explain-plan-on-ddls.html
[2]
https://stackoverflow.com/questions/827123/how-can-i-estimate-the-size-of-an-oracle-index
[3]
https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/EXPLAIN-PLAN.html
[4]
https://docs.oracle.com/en/database/oracle/oracle-database/21/arpls/DBMS_SPACE.html#GUID-B6EB7527-BC67-4394-9E7A-01F2790C409A

Regards,

Sami

--
Guillaume.

#18Sami Imseih
samimseih@gmail.com
In reply to: Guillaume Lelarge (#17)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

You might be interested by this thread "Thinking about EXPLAIN ALTER TABLE":

/messages/by-id/CAM-w4HNm1M5J-ow8UjTcqRe3JPxkVCrGe56tRpPUSePSdGcZ_w@mail.gmail.com

I reviewed this thread, and the primary issue with the EXPLAIN command lies
in the inability to predict all the steps the ALTER TABLE will take as some
are made in phase 2 or 3. It is unlikely that all significant
decisions can be made
in phase 1.

However, I don't think that EXPLAIN addresses the same problem as the proposed
VERBOSE option. Consider, for instance, a user who intends to perform a schema
change that includes a long sequence of ALTERs. These steps will depend on each
other. Tom raises this point of dependent ALTERs [2] but for a
different reason.
I think however, this is an important point. How will EXPLAIN help
here? It may not tell
you the truth, because it does not actually do the work and can't know
the future
state of the schema.

VERBOSE on the other hand will perform the steps, and a user
can test these changes in a test environment or a schema-only restore
and know exactly what to expect in production.

As this thread has expanded beyond its original scope,
I believe this broader discussion is valuable. Rather than merely
addressing the DEBUG1 issue, we can work toward a more complete
and beneficial solution.

Thoughts?

[0]: /messages/by-id/26597.1544460770@sss.pgh.pa.us

Regards,

Sami

#19Sami Imseih
samimseih@gmail.com
In reply to: Sami Imseih (#18)
Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

At this point I am planning on withdrawing this patch
from the March commitfest. I don't think fixing the REINDEX
debug1 output makes a whole lot of sense. I still think more logging
for (CREATE|ALTER) (INDEX|TABLE) will be a good to have but there
needs to be more discussion about the best approach.

Regards,

Sami