pgsql: Implement "fastupdate" support for GIN indexes, in which we try

Started by Tom Laneabout 17 years ago4 messageshackers
Jump to latest
#1Tom Lane
tgl@sss.pgh.pa.us

Log Message:
-----------
Implement "fastupdate" support for GIN indexes, in which we try to accumulate
multiple index entries in a holding area before adding them to the main index
structure. This helps because bulk insert is (usually) significantly faster
than retail insert for GIN.

This patch also removes GIN support for amgettuple-style index scans. The
API defined for amgettuple is difficult to support with fastupdate, and
the previously committed partial-match feature didn't really work with
it either. We might eventually figure a way to put back amgettuple
support, but it won't happen for 8.4.

catversion bumped because of change in GIN's pg_am entry, and because
the format of GIN indexes changed on-disk (there's a metapage now,
and possibly a pending list).

Teodor Sigaev

Modified Files:
--------------
pgsql/doc/src/sgml:
gin.sgml (r2.16 -> r2.17)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/gin.sgml?r1=2.16&r2=2.17)
indexam.sgml (r2.29 -> r2.30)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/indexam.sgml?r1=2.29&r2=2.30)
textsearch.sgml (r1.47 -> r1.48)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/textsearch.sgml?r1=1.47&r2=1.48)
pgsql/doc/src/sgml/ref:
create_index.sgml (r1.70 -> r1.71)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/ref/create_index.sgml?r1=1.70&r2=1.71)
vacuum.sgml (r1.54 -> r1.55)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/ref/vacuum.sgml?r1=1.54&r2=1.55)
pgsql/src/backend/access/common:
reloptions.c (r1.23 -> r1.24)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/common/reloptions.c?r1=1.23&r2=1.24)
pgsql/src/backend/access/gin:
Makefile (r1.3 -> r1.4)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/Makefile?r1=1.3&r2=1.4)
ginbulk.c (r1.14 -> r1.15)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginbulk.c?r1=1.14&r2=1.15)
gindatapage.c (r1.13 -> r1.14)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/gindatapage.c?r1=1.13&r2=1.14)
ginget.c (r1.22 -> r1.23)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginget.c?r1=1.22&r2=1.23)
gininsert.c (r1.18 -> r1.19)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/gininsert.c?r1=1.18&r2=1.19)
ginutil.c (r1.20 -> r1.21)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginutil.c?r1=1.20&r2=1.21)
ginvacuum.c (r1.27 -> r1.28)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginvacuum.c?r1=1.27&r2=1.28)
ginxlog.c (r1.17 -> r1.18)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginxlog.c?r1=1.17&r2=1.18)
pgsql/src/backend/access/gist:
gistvacuum.c (r1.42 -> r1.43)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gist/gistvacuum.c?r1=1.42&r2=1.43)
pgsql/src/backend/access/hash:
hash.c (r1.108 -> r1.109)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/hash/hash.c?r1=1.108&r2=1.109)
pgsql/src/backend/access/index:
indexam.c (r1.112 -> r1.113)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/index/indexam.c?r1=1.112&r2=1.113)
pgsql/src/backend/access/nbtree:
nbtree.c (r1.167 -> r1.168)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/nbtree/nbtree.c?r1=1.167&r2=1.168)
pgsql/src/backend/catalog:
index.c (r1.312 -> r1.313)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/catalog/index.c?r1=1.312&r2=1.313)
pgsql/src/backend/commands:
analyze.c (r1.133 -> r1.134)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/analyze.c?r1=1.133&r2=1.134)
vacuum.c (r1.385 -> r1.386)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/vacuum.c?r1=1.385&r2=1.386)
vacuumlazy.c (r1.118 -> r1.119)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/vacuumlazy.c?r1=1.118&r2=1.119)
pgsql/src/backend/nodes:
tidbitmap.c (r1.17 -> r1.18)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/nodes/tidbitmap.c?r1=1.17&r2=1.18)
pgsql/src/include/access:
genam.h (r1.75 -> r1.76)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/access/genam.h?r1=1.75&r2=1.76)
gin.h (r1.28 -> r1.29)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/access/gin.h?r1=1.28&r2=1.29)
pgsql/src/include/catalog:
catversion.h (r1.524 -> r1.525)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/catalog/catversion.h?r1=1.524&r2=1.525)
pg_am.h (r1.61 -> r1.62)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/catalog/pg_am.h?r1=1.61&r2=1.62)
pg_proc.h (r1.537 -> r1.538)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/catalog/pg_proc.h?r1=1.537&r2=1.538)
pgsql/src/include/nodes:
tidbitmap.h (r1.9 -> r1.10)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/nodes/tidbitmap.h?r1=1.9&r2=1.10)

Added Files:
-----------
pgsql/src/backend/access/gin:
ginfast.c (r1.1)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginfast.c?rev=1.1&content-type=text/x-cvsweb-markup)

#2Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#1)
Re: [COMMITTERS] pgsql: Implement "fastupdate" support for GIN indexes, in which we try

2009/3/24 Tom Lane <tgl@postgresql.org>:

Implement "fastupdate" support for GIN indexes, in which we try to accumulate
multiple index entries in a holding area before adding them to the main index
structure.  This helps because bulk insert is (usually) significantly faster
than retail insert for GIN.

This patch also removes GIN support for amgettuple-style index scans.  The
API defined for amgettuple is difficult to support with fastupdate, and
the previously committed partial-match feature didn't really work with
it either.  We might eventually figure a way to put back amgettuple
support, but it won't happen for 8.4.

catversion bumped because of change in GIN's pg_am entry, and because
the format of GIN indexes changed on-disk (there's a metapage now,
and possibly a pending list).

Will this break pg_migrator?

...Robert

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#2)
Re: Re: [COMMITTERS] pgsql: Implement "fastupdate" support for GIN indexes, in which we try

Robert Haas <robertmhaas@gmail.com> writes:

2009/3/24 Tom Lane <tgl@postgresql.org>:

catversion bumped because of change in GIN's pg_am entry, and because
the format of GIN indexes changed on-disk (there's a metapage now,
and possibly a pending list).

Will this break pg_migrator?

No worse than it's already broken by hash indexes. I would imagine
the strategy will just be to force a REINDEX on any non-btree indexes.

regards, tom lane

#4Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#3)
Re: Re: [COMMITTERS] pgsql: Implement "fastupdate" support for GIN indexes, in which we try

Tom Lane wrote:

Robert Haas <robertmhaas@gmail.com> writes:

2009/3/24 Tom Lane <tgl@postgresql.org>:

catversion bumped because of change in GIN's pg_am entry, and because
the format of GIN indexes changed on-disk (there's a metapage now,
and possibly a pending list).

Will this break pg_migrator?

No worse than it's already broken by hash indexes. I would imagine
the strategy will just be to force a REINDEX on any non-btree indexes.

Agreed.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +