First version of multi-key index support for GiST

Do you want this applied to the current CVS?

We have implemented multi-key index support for GiST. Patch is available
from http://www.sai.msu.su/~megera/postgres/gist/code/7.1.2/patch_multikeygist.7.1.2.gz
The patch could be applied for postgresql version 7.1.2 and current sources 7.2

1. initdb is required. But, it's possible just to execute update
update pg_am set amstrategies = 12 where amname = 'gist';

2. You have to recompile all gist_*_ops functions

3. multi-key indexes works only for ??? gist__int_ops and
gist__intbig_ops (from contrib/intarray), because they have
support for NULLs.

As a bonus we fixed several memory leaks in old GiST code.

Example:

create index mgix on tabletest using gist (b gist_int_ops, a
gist__intbig_ops ) with ( islossy );

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

On Sat, 26 May 2001, Bruce Momjian wrote:

Do you want this applied to the current CVS?

Sure. I want our development to be in sync with cvs
There are several problems we have to resolve but basic functionality
is there and seems works for us.

Oleg

We have implemented multi-key index support for GiST. Patch is available
from http://www.sai.msu.su/~megera/postgres/gist/code/7.1.2/patch_multikeygist.7.1.2.gz
The patch could be applied for postgresql version 7.1.2 and current sources 7.2

1. initdb is required. But, it's possible just to execute update
update pg_am set amstrategies = 12 where amname = 'gist';

2. You have to recompile all gist_*_ops functions

3. multi-key indexes works only for ??? gist__int_ops and
gist__intbig_ops (from contrib/intarray), because they have
support for NULLs.

As a bonus we fixed several memory leaks in old GiST code.

Example:

create index mgix on tabletest using gist (b gist_int_ops, a
gist__intbig_ops ) with ( islossy );

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

Your patch has been added to the PostgreSQL unapplied patches list at:

http://candle.pha.pa.us/cgi-bin/pgpatches

I will try to apply it withing the next 48 hours.

[ Charset KOI8-R unsupported, converting... ]

We have implemented multi-key index support for GiST. Patch is available
from http://www.sai.msu.su/~megera/postgres/gist/code/7.1.2/patch_multikeygist.7.1.2.gz
The patch could be applied for postgresql version 7.1.2 and current sources 7.2

1. initdb is required. But, it's possible just to execute update
update pg_am set amstrategies = 12 where amname = 'gist';

2. You have to recompile all gist_*_ops functions

3. multi-key indexes works only for ??? gist__int_ops and
gist__intbig_ops (from contrib/intarray), because they have
support for NULLs.

As a bonus we fixed several memory leaks in old GiST code.

Example:

create index mgix on tabletest using gist (b gist_int_ops, a
gist__intbig_ops ) with ( islossy );

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Oleg Bartunov <oleg@sai.msu.su> writes:

We have implemented multi-key index support for GiST. Patch is available
from http://www.sai.msu.su/~megera/postgres/gist/code/7.1.2/patch_multikeygist.7.1.2.gz

What is the point of the macro

#define ATTGET(itup, Rel, i, isnull ) ((char*)( \
( IndexTupleSize(itup) == sizeof(IndexTupleData) ) ? \
*(isnull)=true, NULL \
: \
index_getattr(itup, i, (Rel)->rd_att, isnull) \
))

It appears to me that index_getattr should handle an all-NULL index
tuple just fine by itself --- certainly the btree code expects it to.
So I do not see the reason for this extra layer on top of it.

regards, tom lane

What is the point of the macro

#define ATTGET(itup, Rel, i, isnull ) ((char*)( \
( IndexTupleSize(itup) == sizeof(IndexTupleData) ) ? \
*(isnull)=true, NULL \
: \
index_getattr(itup, i, (Rel)->rd_att, isnull) \
))

It appears to me that index_getattr should handle an all-NULL index
tuple just fine by itself --- certainly the btree code expects it to.
So I do not see the reason for this extra layer on top of it.

You are right. It can be removed or replaced to
#define ATTGET(itup, Rel, i, isnull ) (char*)( index_getattr(itup, i, (Rel)->rd_att, isnull) )

The point was that in gist_tuple_replacekey (called from gistPageAddItem) key may be

replaced by null value, but flag itup->t_info & INDEX_NULL_MASK is not set.

Now we don't use gistPageAddItem (

see http://fts.postgresql.org/db/mw/msg.html?mid=118707).

This is our oversight.

--
Teodor Sigaev
teodor@stack.net

Teodor Sigaev <teodor@stack.net> writes:

The point was that in gist_tuple_replacekey (called from
gistPageAddItem) key may be replaced by null value, but flag
itup->t_info & INDEX_NULL_MASK is not set.

Ah. That's certainly a bug.

regards, tom lane

Oleg Bartunov <oleg@sai.msu.su> writes:

We have implemented multi-key index support for GiST. Patch is available
from http://www.sai.msu.su/~megera/postgres/gist/code/7.1.2/patch_multikeygist.7.1.2.gz

I have committed these changes, along with your leak patch of 5/30.

1. initdb is required. But, it's possible just to execute update
update pg_am set amstrategies = 12 where amname = 'gist';

No initdb is needed --- I fixed the code instead ;-)

2. You have to recompile all gist_*_ops functions

I bit the bullet and fixed all the places that were using "char*" where
they should have been using "Datum". This doesn't completely free GIST
from datatype assumptions: it still assumes that all datatypes it deals
with will be pass-by-reference. But it's a step forward. This means
not only a recompile but code changes for any user-supplied GIST ops.
I applied the appropriate changes to everything that's in contrib
(including your new RTREE emulation code).

regards, tom lane

On Thu, 31 May 2001, Tom Lane wrote:

they should have been using "Datum". This doesn't completely free GIST
from datatype assumptions: it still assumes that all datatypes it deals
with will be pass-by-reference. But it's a step forward. This means

I'm afraid this problem is connected with the problem of index_formtuple -
all keys (even btree_ops) are greater than 4 bytes, so it's impossible
to pass them by value. They should be pass-by-reference.
So, probably functions gistindex and gistbuild should be modified for
translation from pass-by-value to pass-by-reference. Your comments ?

regards, tom lane

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83