pgsql: Local partitioned indexes

Started by Alvaro Herreraover 8 years ago8 messageshackerscomitters
Beta feature

Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.

won't retrysuccessCI history

You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:

docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t38049
psql -h localhost -U postgres

Built from patchset v7 (message #7), August 18, 2026 at 04:57 PM.

Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:

git clone --branch t38049_7 https://github.com/hackorum-dev/postgres.git

In a checkout you already have, add the fork once:

git remote add hackorum https://github.com/hackorum-dev/postgres.git

then, for this patchset and every later one:

git fetch hackorum t38049_7 && git checkout t38049_7

Patchset v7 (message #7) is on t38049_7

Jump to latest
#1Alvaro Herrera
alvherre@2ndquadrant.com
hackerscomitters

Local partitioned indexes

When CREATE INDEX is run on a partitioned table, create catalog entries
for an index on the partitioned table (which is just a placeholder since
the table proper has no data of its own), and recurse to create actual
indexes on the existing partitions; create them in future partitions
also.

As a convenience gadget, if the new index definition matches some
existing index in partitions, these are picked up and used instead of
creating new ones. Whichever way these indexes come about, they become
attached to the index on the parent table and are dropped alongside it,
and cannot be dropped on isolation unless they are detached first.

To support pg_dump'ing these indexes, add commands
CREATE INDEX ON ONLY <table>
(which creates the index on the parent partitioned table, without
recursing) and
ALTER INDEX ATTACH PARTITION
(which is used after the indexes have been created individually on each
partition, to attach them to the parent index). These reconstruct prior
database state exactly.

Reviewed-by: (in alphabetical order) Peter Eisentraut, Robert Haas, Amit
Langote, Jesper Pedersen, Simon Riggs, David Rowley
Discussion: /messages/by-id/20171113170646.gzweigyrgg6pwsg4@alvherre.pgsql

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/8b08f7d4820fd7a8ef6152a9dd8c6e3cb01e5f99

Modified Files
--------------
doc/src/sgml/catalogs.sgml | 23 +
doc/src/sgml/ref/alter_index.sgml | 14 +
doc/src/sgml/ref/alter_table.sgml | 8 +-
doc/src/sgml/ref/create_index.sgml | 33 +-
doc/src/sgml/ref/reindex.sgml | 5 +
src/backend/access/common/reloptions.c | 1 +
src/backend/access/heap/heapam.c | 9 +-
src/backend/access/index/indexam.c | 3 +-
src/backend/bootstrap/bootparse.y | 2 +
src/backend/catalog/aclchk.c | 9 +-
src/backend/catalog/dependency.c | 14 +-
src/backend/catalog/heap.c | 1 +
src/backend/catalog/index.c | 203 +++++++-
src/backend/catalog/objectaddress.c | 5 +-
src/backend/catalog/pg_depend.c | 13 +-
src/backend/catalog/pg_inherits.c | 80 ++++
src/backend/catalog/toasting.c | 2 +
src/backend/commands/indexcmds.c | 397 +++++++++++++++-
src/backend/commands/tablecmds.c | 653 +++++++++++++++++++++++---
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 33 +-
src/backend/parser/parse_utilcmd.c | 65 ++-
src/backend/tcop/utility.c | 22 +
src/backend/utils/adt/amutils.c | 3 +-
src/backend/utils/adt/ruleutils.c | 17 +-
src/backend/utils/cache/relcache.c | 39 +-
src/bin/pg_dump/common.c | 107 ++++-
src/bin/pg_dump/pg_dump.c | 102 +++-
src/bin/pg_dump/pg_dump.h | 11 +
src/bin/pg_dump/pg_dump_sort.c | 56 ++-
src/bin/pg_dump/t/002_pg_dump.pl | 95 ++++
src/bin/psql/describe.c | 20 +-
src/bin/psql/tab-complete.c | 34 +-
src/include/catalog/dependency.h | 15 +
src/include/catalog/index.h | 10 +
src/include/catalog/pg_class.h | 1 +
src/include/catalog/pg_inherits_fn.h | 3 +
src/include/commands/defrem.h | 3 +-
src/include/nodes/execnodes.h | 1 +
src/include/nodes/parsenodes.h | 7 +-
src/include/parser/parse_utilcmd.h | 3 +
src/test/regress/expected/alter_table.out | 65 ++-
src/test/regress/expected/indexing.out | 757 ++++++++++++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/alter_table.sql | 16 +
src/test/regress/sql/indexing.sql | 388 +++++++++++++++
49 files changed, 3172 insertions(+), 182 deletions(-)

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#1)
hackerscomitters
Re: pgsql: Local partitioned indexes

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Local partitioned indexes

Evidently you're not there yet. I'm suspicious that the continuing
failures on dromedary may trace to its use of -DCOPY_PARSE_PLAN_TREES
... try looking for a missed field addition in copyfuncs.c.

regards, tom lane

#3Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Tom Lane (#2)
hackerscomitters
Re: pgsql: Local partitioned indexes

Tom Lane wrote:

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Local partitioned indexes

Evidently you're not there yet. I'm suspicious that the continuing
failures on dromedary may trace to its use of -DCOPY_PARSE_PLAN_TREES
... try looking for a missed field addition in copyfuncs.c.

I had already tried COPY_PARSE_PLAN_TREES locally, but that doesn't
reproduce the problem in my machine.
Peter E. noticed that the factor in common in these failures is that the
machines are 32 bits -- so now that I'm back from lunch I can now
reproduce in a 32bit VM and I'm looking into it.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#4Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Alvaro Herrera (#1)
hackerscomitters
Re: pgsql: Local partitioned indexes

On 2018/01/19 23:55, Alvaro Herrera wrote:

Local partitioned indexes

Modified Files
--------------
doc/src/sgml/catalogs.sgml | 23 +
doc/src/sgml/ref/alter_index.sgml | 14 +
doc/src/sgml/ref/alter_table.sgml | 8 +-
doc/src/sgml/ref/create_index.sgml | 33 +-
doc/src/sgml/ref/reindex.sgml | 5 +

I noticed that the declarative partitioning section in ddl.sgml hasn't
been updated to reflect the features added by this commit. Attached patch
is an attempt to fix that.

Thanks,
Amit

Attachments:

v1-0001-Update-ddl.sgml-to-reflect-features-added-in-8b08.patchtext/plain; charset=UTF-8; name=v1-0001-Update-ddl.sgml-to-reflect-features-added-in-8b08.patchDownload+15-7
#5Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Amit Langote (#4)
hackerscomitters
Re: pgsql: Local partitioned indexes

Amit Langote wrote:

On 2018/01/19 23:55, Alvaro Herrera wrote:

Local partitioned indexes

I noticed that the declarative partitioning section in ddl.sgml hasn't
been updated to reflect the features added by this commit. Attached patch
is an attempt to fix that.

Thanks! I considered that keeping the old-style instructions creating
per partition indexes individually was not necessary, so I removed them
and pushed.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#6Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Alvaro Herrera (#5)
hackerscomitters
Re: pgsql: Local partitioned indexes

On Sat, Feb 10, 2018 at 10:09 PM, Alvaro Herrera
<alvherre@alvh.no-ip.org> wrote:

Amit Langote wrote:

On 2018/01/19 23:55, Alvaro Herrera wrote:

Local partitioned indexes

I noticed that the declarative partitioning section in ddl.sgml hasn't
been updated to reflect the features added by this commit. Attached patch
is an attempt to fix that.

Thanks! I considered that keeping the old-style instructions creating
per partition indexes individually was not necessary, so I removed them
and pushed.

Ah, thanks. What you've committed looks perfect.

Regards,
Amit

#7Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Amit Langote (#6)
hackerscomitters
Re: pgsql: Local partitioned indexes

On 2018/02/10 23:32, Amit Langote wrote:

On Sat, Feb 10, 2018 at 10:09 PM, Alvaro Herrera
<alvherre@alvh.no-ip.org> wrote:

Amit Langote wrote:

On 2018/01/19 23:55, Alvaro Herrera wrote:

Local partitioned indexes

I noticed that the declarative partitioning section in ddl.sgml hasn't
been updated to reflect the features added by this commit. Attached patch
is an attempt to fix that.

Thanks! I considered that keeping the old-style instructions creating
per partition indexes individually was not necessary, so I removed them
and pushed.

Ah, thanks. What you've committed looks perfect.

Sorry, I'd missed reporting one more sentence that doesn't apply anymore.
Attached gets rid of that.

Thanks,
Amit

Attachments:

t38049_7
ddl-partition-index.patchtext/plain; charset=UTF-8; name=ddl-partition-index.patchDownload+1-1
#8Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Amit Langote (#7)
hackerscomitters
Re: pgsql: Local partitioned indexes

Amit Langote wrote:

Sorry, I'd missed reporting one more sentence that doesn't apply anymore.
Attached gets rid of that.

Thanks, applied.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services