7.1 features list
Here is the list of features in 7.1.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026Attachments:
We're working hardly on bugfixes for GiST (I've posted patch for 7.0.3)
and probably could finish in 1-2 weeks.
regards,
Oleg
On Sat, 16 Dec 2000, Bruce Momjian wrote:
Date: Sat, 16 Dec 2000 15:16:22 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
To: PostgreSQL-development <pgsql-hackers@postgresql.org>,
PostgreSQL-documentation <pgsql-docs@postgresql.org>
Subject: [HACKERS] 7.1 features listHere is the list of features in 7.1. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
At 3:16 PM -0500 12/16/00, Bruce Momjian wrote:
Here is the list of features in 7.1.
New Darwin/Mac OSX port (Bruce Hartzler)
Not to be a snob, but I probably did 80% of this.
(BTW- tons of stuff at www.postgresql.org is busted. Searching mailing list archives for example.)
-pmb
--
"Every time you provide an option, you're asking the user to make a decision.
That means they will have to think about something and decide about it.
It's not necessarily a bad thing, but, in general, you should always try to
minimize the number of decisions that people have to make."
http://joel.editthispage.com/stories/storyReader$51
Peter Bierman <bierman@apple.com> writes:
At 3:16 PM -0500 12/16/00, Bruce Momjian wrote:
Here is the list of features in 7.1.
New Darwin/Mac OSX port (Bruce Hartzler)
Not to be a snob, but I probably did 80% of this.
Bruce had submitted an earlier patch, but IIRC Peter's version was the
one that got applied. (Or was Peter doing mopup work on Bruce's first
cut? I forget.) At the very least Peter should get 50% credit...
regards, tom lane
Hi,
I checked 7.1 feature list and didn't find any mention about GiST
but there are changes in GiST code. Who is a maintainer of GiST code ?
We have several problems with GiSt and would like to communicate
with somebody who understand these code.
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov <oleg@sai.msu.su> writes:
I checked 7.1 feature list and didn't find any mention about GiST
but there are changes in GiST code. Who is a maintainer of GiST code ?
You are ;-). If you expect to find someone who understands GiST better
than you, you're probably out of luck.
I recall having made a number of changes that applied to all of the
index access methods, including GiST --- but I was just changing
similar code in all the methods. I don't claim to know anything
about GiST in particular.
regards, tom lane
On Sat, 16 Dec 2000, Peter Bierman wrote:
At 3:16 PM -0500 12/16/00, Bruce Momjian wrote:
Here is the list of features in 7.1.
New Darwin/Mac OSX port (Bruce Hartzler)Not to be a snob, but I probably did 80% of this.
(BTW- tons of stuff at www.postgresql.org is busted. Searching mailing
list archives for example.)
Please provide URLs where you are trying to search ... we did extensive
work over the past few weeks to speed up searching, and I tend randomly to
make sure things are still running fine, and haven't had any problems with
either speed or broken links ...
its possible its one of the mirror sites?
At 9:43 PM -0400 12/17/00, The Hermit Hacker wrote:
On Sat, 16 Dec 2000, Peter Bierman wrote:
(BTW- tons of stuff at www.postgresql.org is busted. Searching mailing
list archives for example.)Please provide URLs where you are trying to search ... we did extensive
work over the past few weeks to speed up searching, and I tend randomly to
make sure things are still running fine, and haven't had any problems with
either speed or broken links ...
Clicking on the "search" pic/link at http://www.postgresql.org/ always brought up a dialog (IE5-Mac) that said "the attempt to load http://www.postgresql.org/search.mpl" failed.
But just now I pasted that URL into the location, and it loads fine. And now the pic/link works fine too. I have no idea what's with that.
Just now I went to http://www.postgresql.org/mhonarc/pgsql-hackers/
typed 'foo' in the search field, and I get a dialog a few seconds later:
"The attempt to load:"Accessing URL: http://www.postgresql.org/mhonarc/pgsql-hackers/search.mpl?<stuff>" (runs offscreen).
Maybe it's some javascript that's trying to load a "still loading" page, and has a bogus URL with some explanatory text prepended? (Note the URL:"Accessing URL:http...")
I loaded "http://www.postgresql.org/mhonarc/pgsql-hackers/search.mpl?" by hand, and then went back and tried a search again, and now it works.
Dunno what's going on here. Since it never worked for me, I never tried loading the URL by hand. Obviously it's more complicated than the outright broken link that I thought it was, sorry about that.
-pmb
--
"Every time you provide an option, you're asking the user to make a decision.
That means they will have to think about something and decide about it.
It's not necessarily a bad thing, but, in general, you should always try to
minimize the number of decisions that people have to make."
http://joel.editthispage.com/stories/storyReader$51
Yes, I will take a pass over the logs before final. As noted in the
file, the list is accurate as of December 11.
We're working hardly on bugfixes for GiST (I've posted patch for 7.0.3)
and probably could finish in 1-2 weeks.regards,
Oleg
On Sat, 16 Dec 2000, Bruce Momjian wrote:Date: Sat, 16 Dec 2000 15:16:22 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
To: PostgreSQL-development <pgsql-hackers@postgresql.org>,
PostgreSQL-documentation <pgsql-docs@postgresql.org>
Subject: [HACKERS] 7.1 features listHere is the list of features in 7.1. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026Peter Bierman <bierman@apple.com> writes:
At 3:16 PM -0500 12/16/00, Bruce Momjian wrote:
Here is the list of features in 7.1.
New Darwin/Mac OSX port (Bruce Hartzler)Not to be a snob, but I probably did 80% of this.
Bruce had submitted an earlier patch, but IIRC Peter's version was the
one that got applied. (Or was Peter doing mopup work on Bruce's first
cut? I forget.) At the very least Peter should get 50% credit...
Modified entry. Thanks for the correction:
New Darwin/Mac OSX port (Peter Bierman, Bruce Hartzler)
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026Oleg Bartunov <oleg@sai.msu.su> writes:
I checked 7.1 feature list and didn't find any mention about GiST
but there are changes in GiST code. Who is a maintainer of GiST code ?You are ;-). If you expect to find someone who understands GiST better
than you, you're probably out of luck.I recall having made a number of changes that applied to all of the
index access methods, including GiST --- but I was just changing
similar code in all the methods. I don't claim to know anything
about GiST in particular.
I know I met someone who said they invented Gist. Tom, was that at the
Database Summit? I can't think of that person's name now. I think
there are some papers at Berkeley or a web site that goes into it in
detail.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026Bruce Momjian <pgman@candle.pha.pa.us> writes:
I think
there are some papers at Berkeley or a web site that goes into it in
detail.
I imagine there's some GiST stuff at the Berkeley papers repository
http://s2k-ftp.CS.Berkeley.EDU:8000/postgres/papers/
but I'd be surprised if it's more than an overview...
regards, tom lane
Peter Bierman <bierman@apple.com> writes:
Just now I went to http://www.postgresql.org/mhonarc/pgsql-hackers/
typed 'foo' in the search field, and I get a dialog a few seconds later:
"The attempt to load:"Accessing URL: http://www.postgresql.org/mhonarc/pgsql-hackers/search.mpl?<stuff>" (runs offscreen).
Odd, the same experiment seems to work fine for me. Maybe a browser
dependency? I'm using Netscape 4.75 on HPUX ...
Maybe it's some javascript
I don't see any javascript on the loaded page.
regards, tom lane
On Mon, 18 Dec 2000, Tom Lane wrote:
Peter Bierman <bierman@apple.com> writes:
Just now I went to http://www.postgresql.org/mhonarc/pgsql-hackers/
typed 'foo' in the search field, and I get a dialog a few seconds later:
"The attempt to load:"Accessing URL: http://www.postgresql.org/mhonarc/pgsql-hackers/search.mpl?<stuff>" (runs offscreen).
Odd, the same experiment seems to work fine for me. Maybe a browser
dependency? I'm using Netscape 4.75 on HPUX ...
Just went to the above URL using IE 5.5, types in 'foo' and it came back
with 909 matches found ...
Maybe it's some javascript
I don't see any javascript on the loaded page.
none used that I'm aware of either ...
regards, tom lane
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
OK, now I understand the situation. Another question:
Who is a maintainer of Rtree code ? We have a problem with
handling NULL values in GiST. Any thought how NULL values
are handle in Rtree.
regards,
Oleg
On Sun, 17 Dec 2000, Bruce Momjian wrote:
Date: Sun, 17 Dec 2000 23:23:32 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Oleg Bartunov <oleg@sai.msu.su>,
PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Who is a maintainer of GiST code ?Oleg Bartunov <oleg@sai.msu.su> writes:
I checked 7.1 feature list and didn't find any mention about GiST
but there are changes in GiST code. Who is a maintainer of GiST code ?You are ;-). If you expect to find someone who understands GiST better
than you, you're probably out of luck.I recall having made a number of changes that applied to all of the
index access methods, including GiST --- but I was just changing
similar code in all the methods. I don't claim to know anything
about GiST in particular.I know I met someone who said they invented Gist. Tom, was that at the
Database Summit? I can't think of that person's name now. I think
there are some papers at Berkeley or a web site that goes into it in
detail.-- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov <oleg@sai.msu.su> writes:
We have a problem with
handling NULL values in GiST. Any thought how NULL values
are handle in Rtree.
AFAIR, none of the index access methods except btree handle NULLs at
all --- they just ignore NULL values and don't store them in the index.
Feel free to improve on that ;-). The physical representation of index
tuples can handle NULLs, the problem is teaching the index logic where
they should go in the index.
regards, tom lane
Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
We have a problem with
handling NULL values in GiST. Any thought how NULL values
are handle in Rtree.AFAIR, none of the index access methods except btree handle NULLs at
all --- they just ignore NULL values and don't store them in the index.
Feel free to improve on that ;-). The physical representation of index
tuples can handle NULLs, the problem is teaching the index logic where
they should go in the index.regards, tom lane
and I can't see why btree stores them (as it seems to do judging by the
index file size) - at least it does not use it for searching for "IS
NULL"
--8<--------8<--------8<--------8<--------8<--------8<--------8<--------8<------
hannu=# explain select * from nulltest where i is null;
NOTICE: QUERY PLAN:
Seq Scan on nulltest (cost=0.00..293.80 rows=5461 width=8)
EXPLAIN
hannu=# explain select * from nulltest where i =1;
NOTICE: QUERY PLAN:
Index Scan using nulltest_i_ndx on nulltest (cost=0.00..96.95 rows=164
width=8)
--8<--------8<--------8<--------8<--------8<--------8<--------8<--------8<------
nulltest is a 16k record table with numbers 1 to 16384 in field i
If it just ignored them we would have a nice way to fake partial indexes
-
just define a function that returns field value or null and then index
on that ;)
-----------
Hannu
Hannu Krosing <hannu@tm.ee> writes:
and I can't see why btree stores them (as it seems to do judging by the
index file size) - at least it does not use it for searching for "IS
NULL"
That's another thing that needs improvement ;-). Seems to me it should
be able to do that.
The reason why btree *has* to be able to deal with null entries is to
cope with multi-column indexes; you don't want it refusing to index a
row at all just because some of the columns are null. The others don't
currently handle multi-column indexes, so they're not really forced
to deal with that issue.
From a purely semantic point of view I'm not sure why Oleg is worried
about being able to store nulls in a GiST index ... seems like leaving
them out is OK, modulo the occasional complaint from VACUUM's
insufficiently intelligent tuple-count comparison ...
regards, tom lane
On Sat, 16 Dec 2000, Bruce Momjian wrote:
Here is the list of features in 7.1.
One thing that I think ought to be added is that with 7.1,
PostgreSQL will compile out of the box (i.e. without any extra patches)
for Linux/Alpha. This might not be a big deal for most people, but for
those of who run pgsql on Linux/Alpha, it is, and I feel it at least
deserves a mention in the 7.1 feature list.
I looked for it (i.e. grep -i alpha) in the list, but did not see
it. Your choice which heading it goes under.
Also, I have not tested any recent snapshots or betas on
Linux/Alpha lately, but I plan to shortly and will let the hackers list
know of any problems. I have every intention of making sure the 7.1
release does indeed work out of box on Linux/Alpha. Thanks, TTYL.
---------------------------------------------------------------------------
| "For to me to live is Christ, and to die is gain." |
| --- Philippians 1:21 (KJV) |
---------------------------------------------------------------------------
| Ryan Kirkpatrick | Boulder, Colorado | http://www.rkirkpat.net/ |
---------------------------------------------------------------------------
On Sun, Dec 17, 2000 at 11:30:23PM -0500, Tom Lane wrote:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
I think
there are some papers at Berkeley or a web site that goes into it in
detail.I imagine there's some GiST stuff at the Berkeley papers repository
http://s2k-ftp.CS.Berkeley.EDU:8000/postgres/papers/
but I'd be surprised if it's more than an overview...
Well, there's this: http://gist.cs.berkeley.edu/
and this: http://gist.cs.berkeley.edu/pggist/
--
Christopher Masto Senior Network Monkey NetMonger Communications
chris@netmonger.net info@netmonger.net http://www.netmonger.net
Free yourself, free your machine, free the daemon -- http://www.freebsd.org/
On Tue, 19 Dec 2000, Hannu Krosing wrote:
Date: Tue, 19 Dec 2000 02:04:02 +0200
From: Hannu Krosing <hannu@tm.ee>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Oleg Bartunov <oleg@sai.msu.su>,
Bruce Momjian <pgman@candle.pha.pa.us>,
PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Who is a maintainer of GiST code ?Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
We have a problem with
handling NULL values in GiST. Any thought how NULL values
are handle in Rtree.AFAIR, none of the index access methods except btree handle NULLs at
all --- they just ignore NULL values and don't store them in the index.
Feel free to improve on that ;-). The physical representation of index
tuples can handle NULLs, the problem is teaching the index logic where
they should go in the index.regards, tom lane
and I can't see why btree stores them (as it seems to do judging by the
index file size) - at least it does not use it for searching for "IS
NULL"
and what does this error means ?
create table rtree_test ( r box );
copy rtree_test from stdin;
\N
\N
\N
\N
........ total 10,000 NULLS
\.
create index rtree_test_idx on rtree_test using rtree ( r );
--ERROR: floating point exception! The last floating point operation either exceeded legal ranges or was a divide by zero
seems rtree doesn't ignore NULL ?
Regards,
Oleg
--8<--------8<--------8<--------8<--------8<--------8<--------8<--------8<------
hannu=# explain select * from nulltest where i is null;
NOTICE: QUERY PLAN:Seq Scan on nulltest (cost=0.00..293.80 rows=5461 width=8)
EXPLAIN
hannu=# explain select * from nulltest where i =1;
NOTICE: QUERY PLAN:Index Scan using nulltest_i_ndx on nulltest (cost=0.00..96.95 rows=164
width=8)--8<--------8<--------8<--------8<--------8<--------8<--------8<--------8<------
nulltest is a 16k record table with numbers 1 to 16384 in field i
If it just ignored them we would have a nice way to fake partial indexes
-
just define a function that returns field value or null and then index
on that ;)-----------
Hannu
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
On Tue, 19 Dec 2000, Christopher Masto wrote:
Date: Tue, 19 Dec 2000 13:33:58 -0500
From: Christopher Masto <chris@netmonger.net>
To: Tom Lane <tgl@sss.pgh.pa.us>, Bruce Momjian <pgman@candle.pha.pa.us>
Cc: Oleg Bartunov <oleg@sai.msu.su>,
PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Who is a maintainer of GiST code ?On Sun, Dec 17, 2000 at 11:30:23PM -0500, Tom Lane wrote:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
I think
there are some papers at Berkeley or a web site that goes into it in
detail.I imagine there's some GiST stuff at the Berkeley papers repository
http://s2k-ftp.CS.Berkeley.EDU:8000/postgres/papers/
but I'd be surprised if it's more than an overview...Well, there's this: http://gist.cs.berkeley.edu/
and this: http://gist.cs.berkeley.edu/pggist/
Thanks,
we do know this sites. We're working on implementation of
RD (Russian Doll) Tree using GiST interface. Current GiST sources
have some bugs, some of them we already fixed and currently we're a
working with handling of NULL values. We're getting broken index for
data with NULLs. btw, how many people use GiST ? It would be nice
to test our changes after we solve our problems.
Regards,
Oleg
--
Christopher Masto Senior Network Monkey NetMonger Communications
chris@netmonger.net info@netmonger.net http://www.netmonger.netFree yourself, free your machine, free the daemon -- http://www.freebsd.org/
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
AFAIR, none of the index access methods except btree
handle NULLs at all --- they just ignore NULL values
and don't store them in the index.
...
and what does this error means ?
create table rtree_test ( r box );
copy rtree_test from stdin;
\N
........ total 10,000 NULLS
\.create index rtree_test_idx on rtree_test using rtree ( r );
--ERROR: floating point exception! The last floating point
operation either exceeded legal ranges or was a divide by zeroseems rtree doesn't ignore NULL ?
No, it doesn't. As well as GiST. Only hash ignores them.
And there is no code in GiST & rtree that take care about NULL
keys. It's probably ok for GiST which is "meta-index" -
index/type methods implementator should decide how to handle NULLs.
As for rtree - seems it's better to ignore NULLs as we did before
for single key btree: rtree is just variation of it.
Vadim
Import Notes
Resolved by subject fallback
I added (Alpha) next to the mention of 64-bit CPUs on the Function
Manager section at the top.
On Sat, 16 Dec 2000, Bruce Momjian wrote:
Here is the list of features in 7.1.
One thing that I think ought to be added is that with 7.1,
PostgreSQL will compile out of the box (i.e. without any extra patches)
for Linux/Alpha. This might not be a big deal for most people, but for
those of who run pgsql on Linux/Alpha, it is, and I feel it at least
deserves a mention in the 7.1 feature list.
I looked for it (i.e. grep -i alpha) in the list, but did not see
it. Your choice which heading it goes under.
Also, I have not tested any recent snapshots or betas on
Linux/Alpha lately, but I plan to shortly and will let the hackers list
know of any problems. I have every intention of making sure the 7.1
release does indeed work out of box on Linux/Alpha. Thanks, TTYL.---------------------------------------------------------------------------
| "For to me to live is Christ, and to die is gain." |
| --- Philippians 1:21 (KJV) |
---------------------------------------------------------------------------
| Ryan Kirkpatrick | Boulder, Colorado | http://www.rkirkpat.net/ |
---------------------------------------------------------------------------
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026Oleg Bartunov <oleg@sai.msu.su> writes:
seems rtree doesn't ignore NULL ?
Hm, maybe not. There are explicit tests to ignore null inputs in hash
indexes (hash/hash.c), and I'd just sort of assumed that rtree and gist
do the same.
FWIW, your example doesn't seem to provoke an error in current sources;
but it does take quite a long time (far longer than building a btree
index on 10000 nulls). That makes me think that indexing nulls in rtree
might be a bad idea even if it works.
regards, tom lane
We finished (cross fingers) our changes in GiST code for 7.0.3
and in next 2 days we plan to send patches for upcoming 7.1 release.
Isn't this too late ? What else we need to submit ?
We have int4array contribution package which added index support for
integer arrays and it's probably should come to contrib directory.
We got about 3 orders of magnitude speedup using RD-Tree.
Probably we need to add regression test for GiST.
We have following fixes for GiST (7.0.3):
1. indices now store on disk in compressed form - was decompressed which
causes broken index when using compression function
(now compression of indices is really works)
2. NULLs now don't broken index
3. added support for keys with variable length - was fixed-length
Patches and contribution package could be prepared in several days,
documentation with some benchmarks and blurbs require more time.
We plan to use these last changes to speedup our full-text-search
( killer application ) and write article to popularize GiST+Postgres.
Any thoughts ?
Regards,
Oleg
On Wed, 20 Dec 2000, Tom Lane wrote:
Date: Wed, 20 Dec 2000 15:00:21 -0500
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Oleg Bartunov <oleg@sai.msu.su>
Cc: Hannu Krosing <hannu@tm.ee>, Bruce Momjian <pgman@candle.pha.pa.us>,
PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Who is a maintainer of GiST code ?Oleg Bartunov <oleg@sai.msu.su> writes:
seems rtree doesn't ignore NULL ?
Hm, maybe not. There are explicit tests to ignore null inputs in hash
indexes (hash/hash.c), and I'd just sort of assumed that rtree and gist
do the same.FWIW, your example doesn't seem to provoke an error in current sources;
but it does take quite a long time (far longer than building a btree
index on 10000 nulls). That makes me think that indexing nulls in rtree
might be a bad idea even if it works.regards, tom lane
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov <oleg@sai.msu.su> writes:
Probably we need to add regression test for GiST.
That would be nice, but let's not hold up the patches for it.
regards, tom lane
We finished (cross fingers) our changes in GiST code for 7.0.3
and in next 2 days we plan to send patches for upcoming 7.1 release.
Isn't this too late ? What else we need to submit ?
Not too late, especially if it is fixing bugs. And even if not, anything
we can do to encourage active development and use of GiST is worth the
effort imho. I'll guess that unless we are talking about some extreme,
far-reaching changes in the backend others will agree.
We have int4array contribution package which added index support for
integer arrays and it's probably should come to contrib directory.
We got about 3 orders of magnitude speedup using RD-Tree.
Probably we need to add regression test for GiST.
We have following fixes for GiST (7.0.3):
1. indices now store on disk in compressed form - was decompressed which
causes broken index when using compression function
(now compression of indices is really works)
2. NULLs now don't broken index
3. added support for keys with variable length - was fixed-length
Patches and contribution package could be prepared in several days,
documentation with some benchmarks and blurbs require more time.
We plan to use these last changes to speedup our full-text-search
( killer application ) and write article to popularize GiST+Postgres.
Any thoughts ?
Go go go !!! :)
- Thomas
Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
seems rtree doesn't ignore NULL ?
Hm, maybe not. There are explicit tests to ignore null inputs in hash
indexes (hash/hash.c), and I'd just sort of assumed that rtree and gist
do the same.FWIW, your example doesn't seem to provoke an error in current sources;
but it does take quite a long time (far longer than building a btree
index on 10000 nulls). That makes me think that indexing nulls in rtree
might be a bad idea even if it works.
Or maybe just some optimisations done for large number of similar keys (
probabilistic page-splitting or some such ;) in btree are not done in
rtree ?
----------
Hannu
Oleg Bartunov wrote:
We finished (cross fingers) our changes in GiST code for 7.0.3
Are tha patches for 7.0.3 already available ?
and in next 2 days we plan to send patches for upcoming 7.1 release.
Isn't this too late ? What else we need to submit ?
We have int4array contribution package which added index support for
integer arrays
what exactly does it mean ?
can i now query for an int4array which contains number 42 or just
an int array that = '{1,2,3}' with index support ?
and it's probably should come to contrib directory.
We got about 3 orders of magnitude speedup using RD-Tree.
Probably we need to add regression test for GiST.
Some samples will do fine for start ;)
We have following fixes for GiST (7.0.3):
1. indices now store on disk in compressed form - was decompressed which
causes broken index when using compression function
(now compression of indices is really works)
2. NULLs now don't broken index
3. added support for keys with variable length - was fixed-lengthPatches and contribution package could be prepared in several days,
documentation with some benchmarks and blurbs require more time.
We plan to use these last changes to speedup our full-text-search
( killer application ) and write article to popularize GiST+Postgres.
Any thoughts ?
Great!
-------
Hannu
On Thu, 21 Dec 2000, Hannu Krosing wrote:
Date: Thu, 21 Dec 2000 11:12:41 +0200
From: Hannu Krosing <hannu@tm.ee>
To: Oleg Bartunov <oleg@sai.msu.su>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Bruce Momjian <pgman@candle.pha.pa.us>,
PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Who is a maintainer of GiST code ?Oleg Bartunov wrote:
We finished (cross fingers) our changes in GiST code for 7.0.3
Are tha patches for 7.0.3 already available ?
yes, we could post it right now
and in next 2 days we plan to send patches for upcoming 7.1 release.
Isn't this too late ? What else we need to submit ?
We have int4array contribution package which added index support for
integer arrayswhat exactly does it mean ?
can i now query for an int4array which contains number 42 or just
an int array that = '{1,2,3}' with index support ?
yes, you need to apply out patch to GiST and functions for
RD-tree. I'll send you in private mail to conserve bandwidth.
It would be great if you test our code in your application.
Regards,
Oleg
-------
Hannu
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov wrote:
On Thu, 21 Dec 2000, Hannu Krosing wrote:
Date: Thu, 21 Dec 2000 11:12:41 +0200
From: Hannu Krosing <hannu@tm.ee>
To: Oleg Bartunov <oleg@sai.msu.su>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Bruce Momjian <pgman@candle.pha.pa.us>,
PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Who is a maintainer of GiST code ?Oleg Bartunov wrote:
We finished (cross fingers) our changes in GiST code for 7.0.3
Are tha patches for 7.0.3 already available ?
yes, we could post it right now
Great!
and in next 2 days we plan to send patches for upcoming 7.1 release.
Isn't this too late ? What else we need to submit ?
We have int4array contribution package which added index support for
integer arrayswhat exactly does it mean ?
can i now query for an int4array which contains number 42 or just
an int array that = '{1,2,3}' with index support ?yes, you need to apply out patch to GiST and functions for
RD-tree. I'll send you in private mail to conserve bandwidth.
It would be great if you test our code in your application.
Thanks, I'm looking forwad to testing it.
---------------
Hannu
Hi,
we've almost totally rewrite gist.c because old code and algorithm
were not suitable for variable size keys. I think it might be
submitted into 7.1 beta source tree. We have fixed several bugs and
memory leaks. Version for 7.0.3 is also available.
Sampe application for contrib area - implementation RD-Tree and index support
for int arrays I'll submit later (Need some documentation).
Regards,
Oleg
Here is a README.gist
--------------------------------
New version of gist.c for PostgreSQL 7.1
New feature:
1. Support of variable size keys - new algorithm of insertion to tree
(GLI - gist layrered insertion). Previous algorithm was implemented
as described in paper by Joseph M. Hellerstein et.al
"Generalized Search Trees for Database Systems". This (old)
algorithm was not suitable for variable size keys and could be
not effective ( walking up-down ) in case of multiple levels split
Bug fixed:
1. fixed bug in gistPageAddItem - key values were written to disk
uncompressed. This caused failure if decompression function
does real job.
2. NULLs handling - we keep NULLs in tree. Right way is to remove them,
but we don't know how to inform vacuum about index statistics. This is
just cosmetic warning message (like in case with R-Tree),
but I'm not sure how to recognize real problem if we remove NULLs
and suppress this warning as Tom suggested.
3. various memory leaks
All our tests and Gene Selcov's regression tests passed ok.
We have version also for 7.0.3
Sample application which utilize RD-Tree for index support of
int arrays is in contrib/intarray (will be submitted separately).
TODO:
1. Description of GLI algorithm
2. regression test for GiST (based on RD-Tree)
This work was done by Teodor Sigaev (teodor@stack.net) and
Oleg Bartunov (oleg@sai.msu.su).
On Sun, 17 Dec 2000, Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
I checked 7.1 feature list and didn't find any mention about GiST
but there are changes in GiST code. Who is a maintainer of GiST code ?You are ;-). If you expect to find someone who understands GiST better
than you, you're probably out of luck.I recall having made a number of changes that applied to all of the
index access methods, including GiST --- but I was just changing
similar code in all the methods. I don't claim to know anything
about GiST in particular.regards, tom lane
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Attachments:
Oleg Bartunov <oleg@sai.msu.su> writes:
we've almost totally rewrite gist.c because old code and algorithm
were not suitable for variable size keys. I think it might be
submitted into 7.1 beta source tree.
Urgh. Dropping in a total rewrite when we're already past beta3 doesn't
strike me as good project management practice --- especially if the
rewrite was done to add features (ie variable-size keys) not merely
fix bugs. I think it might be more prudent to hold this for 7.2.
regards, tom lane
On Wed, 10 Jan 2001, Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
we've almost totally rewrite gist.c because old code and algorithm
were not suitable for variable size keys. I think it might be
submitted into 7.1 beta source tree.Urgh. Dropping in a total rewrite when we're already past beta3 doesn't
strike me as good project management practice --- especially if the
rewrite was done to add features (ie variable-size keys) not merely
fix bugs. I think it might be more prudent to hold this for 7.2.
OK. If our changes will not go to 7.1, is't possible to create
feature archive and announce it somewhere. It would be nice if
people could test it. Anyway, I'll create web page with all
docs and patches. I'm afraid one more year to 7.2 is enough for
GiST to die :-)
regards, tom lane
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
we've almost totally rewrite gist.c because old code and algorithm
were not suitable for variable size keys. I think it might be
submitted into 7.1 beta source tree.Urgh. Dropping in a total rewrite when we're already past
beta3 doesn't
strike me as good project management practice --- especially if the
rewrite was done to add features (ie variable-size keys) not merely
fix bugs. I think it might be more prudent to hold this for 7.2.OK. If our changes will not go to 7.1, is't possible to create
feature archive and announce it somewhere. It would be nice if
people could test it. Anyway, I'll create web page with all
docs and patches. I'm afraid one more year to 7.2 is enough for
GiST to die :-)
I think featureism is the the most prominent argument for PostgreSQL.
Thus standing before a decision to eighter fix GiST bugs and risc a new
bug (limited to GiST) because of an added feature or shipping a known
broken GiST, my vote would definitely be to add Oleg's patch.
Andreas
Import Notes
Resolved by subject fallback
On Thu, 11 Jan 2001, Zeugswetter Andreas SB wrote:
we've almost totally rewrite gist.c because old code and algorithm
were not suitable for variable size keys. I think it might be
submitted into 7.1 beta source tree.Urgh. Dropping in a total rewrite when we're already past
beta3 doesn't
strike me as good project management practice --- especially if the
rewrite was done to add features (ie variable-size keys) not merely
fix bugs. I think it might be more prudent to hold this for 7.2.OK. If our changes will not go to 7.1, is't possible to create
feature archive and announce it somewhere. It would be nice if
people could test it. Anyway, I'll create web page with all
docs and patches. I'm afraid one more year to 7.2 is enough for
GiST to die :-)I think featureism is the the most prominent argument for PostgreSQL.
Thus standing before a decision to eighter fix GiST bugs and risc a new
bug (limited to GiST) because of an added feature or shipping a known
broken GiST, my vote would definitely be to add Oleg's patch.
Definetely, our changes limited to GiST insert algorithm only.
Other changes are bugfixes. I encourage people interested in GiST
to test my submission. Our implementation of RD-Tree which we used
to support of indexing of int4 arrays will works only with our
version of gist.c (actually our interest to GiST was motivated by
index support of int4 arrays).
Regards,
Oleg
Andreas
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Oleg ... how about a contrib/patches directory that we put this into for
v7.1 release, so that ppl have access to it, and then we apply the patch
first thing for part of v7.2?
On Thu, 11 Jan 2001, Oleg Bartunov wrote:
On Wed, 10 Jan 2001, Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
we've almost totally rewrite gist.c because old code and algorithm
were not suitable for variable size keys. I think it might be
submitted into 7.1 beta source tree.Urgh. Dropping in a total rewrite when we're already past beta3 doesn't
strike me as good project management practice --- especially if the
rewrite was done to add features (ie variable-size keys) not merely
fix bugs. I think it might be more prudent to hold this for 7.2.OK. If our changes will not go to 7.1, is't possible to create
feature archive and announce it somewhere. It would be nice if
people could test it. Anyway, I'll create web page with all
docs and patches. I'm afraid one more year to 7.2 is enough for
GiST to die :-)regards, tom lane
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
Hannu Krosing <hannu@tm.ee> writes:
That's my vote too, specially if there will be some regression tests
accompanying the patches. The current (pre-patch) state of affairs with
GiST could probably be described as security-by-obscurity anyhow i.e.
"we have't tried it so we think it probably works" ;-)
Au contraire, there *are* a few users of GiST out there now, Gene Selkov
to name one. So there is a definite risk of breaking things that worked
in 7.0 and before, in the name of adding new features.
If I thought that we had adequate ability to test the new GiST
implementation during the remaining beta period, I wouldn't be
so worried. But at this point, Oleg's changes could not appear
in the beta series before beta4, and between the late date, the
lack of regression test, and the few interested people to test it,
I doubt that we'll get any useful coverage.
I would recommend that Oleg do like Ryan K. did for awhile with the
Alpha patches: make them available as a set of diffs to be applied
to the official distribution. We'll be happy to merge them in for
7.2, but the calendar says it's too late for 7.1.
regards, tom lane
Import Notes
Reply to msg id not found: 3A5DDA11.B976E4E3@tm.ee
Hannu Krosing <hannu@tm.ee> writes:
... the calendar says it's too late for 7.1.
Even for the _real_ bugfixes in gist.c ?
If he were submitting only bugfixes, we wouldn't be having this
discussion.
Look, I don't like postponing improvements either. But if we don't
adhere to project management discipline, we are never going to get
releases out the door at all --- or if we do, they'll be too buggy
to be reliable. It's not like "no new features during beta" is such
a draconian or difficult-to-understand rule.
The RelFileNodeEquals() bug we found on Monday proves that no one had
yet done enough stress-testing on 7.1 to discover that multiple
databases were broken. Think about that for awhile before you campaign
for inserting untested new features at this point. We need to focus on
TESTING, people, not new features.
regards, tom lane
Import Notes
Reply to msg id not found: 3A5DEED0.DD5742BC@tm.ee
Oleg Bartunov wrote:
On Thu, 11 Jan 2001, Zeugswetter Andreas SB wrote:
I think featureism is the the most prominent argument for PostgreSQL.
Exactly! Altough it has already lost much of it ;(
Thus standing before a decision to eighter fix GiST bugs and risc a new
bug (limited to GiST) because of an added feature or shipping a known
broken GiST, my vote would definitely be to add Oleg's patch.
That's my vote too, specially if there will be some regression tests
accompanying the patches. The current (pre-patch) state of affairs with
GiST could probably be described as security-by-obscurity anyhow i.e.
"we have't tried it so we think it probably works" ;-)
Also I suspect there are still only a few users, most of them capable of
fixing inside gist.c if something nasti turns up.
They will be much more motivated to do so if it is in "official" sources
and not in the sources from postgresql-gist.org or somesuch ;)
Definetely, our changes limited to GiST insert algorithm only.
Other changes are bugfixes.
So applying _only_ bugfixes may also not be an option as they are not
well tested without the changed gist.c
-----------------
Hannu
The Hermit Hacker wrote:
Oleg ... how about a contrib/patches directory that we put this into for
v7.1 release, so that ppl have access to it, and then we apply the patch
first thing for part of v7.2?
And have Mandrake ship postgresql-v7.1-GiST-1mdk.rpm by default ;)
I would even vote for including the ability to index int(4) arrays in
the
main distribution and not in contrib, similar to the current state of
plpgsq
and other pl* - ie they are compiled by default but not "activated".
Assumption that arrays are indexable seems to come up once or twice a
month on the mailing lists.
-------------------
Hannu
The RelFileNodeEquals() bug we found on Monday proves that no one had
yet done enough stress-testing on 7.1 to discover that multiple
databases were broken. Think about that for awhile before
you campaign for inserting untested new features at this point.
We need to focus on TESTING, people, not new features.
I mostly sure that Oleg' changes touch *only* gist subdir (Oleg?)
so *nothing* will be broken in other areas. That's why I don't
object new gist in 7.1.
RelFileNodeEquals is quite another thing, thanks for fix again -:)
Vadim
Import Notes
Resolved by subject fallback
Tom Lane wrote:
Hannu Krosing <hannu@tm.ee> writes:
That's my vote too, specially if there will be some regression tests
accompanying the patches. The current (pre-patch) state of affairs with
GiST could probably be described as security-by-obscurity anyhow i.e.
"we have't tried it so we think it probably works" ;-)Au contraire, there *are* a few users of GiST out there now, Gene Selkov
to name one.
Yes, he is the only one (except Oleg) whom I know to use it too ;)
So there is a definite risk of breaking things that worked
in 7.0 and before, in the name of adding new features.
True. Could we ask Gene to test 7.1 with Oleg's patches ?
If I thought that we had adequate ability to test the new GiST
implementation during the remaining beta period, I wouldn't be
so worried. But at this point, Oleg's changes could not appear
in the beta series before beta4, and between the late date, the
lack of regression test, and the few interested people to test it,
I doubt that we'll get any useful coverage.
Or if in fact there _are_ only a few people using it now we could
get _all_ the coverage to be sufficiently sure we don't break anyones
code. GiST being such an obscure and underused feature I'm pretty sure
that most (all?) active users are on Hackers list and read everything
that has GiST in subject.
I would recommend that Oleg do like Ryan K. did for awhile with the
Alpha patches: make them available as a set of diffs to be applied
to the official distribution. We'll be happy to merge them in for
7.2, but the calendar says it's too late for 7.1.
Even for the _real_ bugfixes in gist.c ?
----------------
Hannu
On Thu, 11 Jan 2001, Mikheev, Vadim wrote:
The RelFileNodeEquals() bug we found on Monday proves that no one had
yet done enough stress-testing on 7.1 to discover that multiple
databases were broken. Think about that for awhile before
you campaign for inserting untested new features at this point.
We need to focus on TESTING, people, not new features.I mostly sure that Oleg' changes touch *only* gist subdir (Oleg?)
Yes, and only one file - gist.c
so *nothing* will be broken in other areas. That's why I don't
object new gist in 7.1.
We prepare regression test for RD-Tree in the same way as Gene
does for his contribution. I put all files on
http://www.sai.msu.su/~megera/postgres/gist/. btw, all Gene's
test for seg and cube in contrib area are passed. It would be better
Gene check his application himself.
I'm sorry for trouble with my submission - I hoped we will be ready
before beta2,3, but we spent too many time to get old insertion
algoritm works with variable size keys until we realized it's just
not suitable for this.
I understand Tom's arguments and respect his experience, so I think it's
possible to put link to my page in 7.1 docs for people interested in
GiST features. Also, we found GiST part of postgres documentation
is too short, so we'll try to contribute something sometime later.
From other side, GiST was too hidden for people, while it's very
powerfull feature and many people for sure really needs GiST power.
Frankly speaking I discovered GiST power myself by accident :-)
Now we have many plans to use GiST in our real life applications such as
Web site management system, full text search (killer application !),
data mining and others.
There are several improvements and new features we plan to add to GiST
which could be go to 7.2.
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
On Thu, 11 Jan 2001, Hannu Krosing wrote:
I make a personal promise to spend at least 5 hours of testing new GiST
functionality during this weekend if it is commited to 7.1 CVS.
(ok, I do it anyhow, just that currently I'm testing it using the
patches ;)
Hanny,
latest version is available at http://www.sai.msu.su/~megera/postgres/gist/
nothing changed in code (in compare with my submission), just added some
info and regression test. Let me know if you need some help.
Oleg
-------------------------
Hannu
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Import Notes
Reply to msg id not found: 3A5E2129.706C9633@tm.ee | Resolved by subject fallback
Oleg Bartunov <oleg@sai.msu.su> writes:
I understand Tom's arguments and respect his experience, so I think it's
possible to put link to my page in 7.1 docs for people interested in
GiST features.
Bear in mind that I only have one core vote ;-). We've already had some
private core discussion about whether to accept this patch now, and so
far I think I'm outvoted.
Did I understand you to say that you'd added some regression tests for
GiST? That would lessen my unhappiness a little bit ...
regards, tom lane
Tom Lane wrote:
Hannu Krosing <hannu@tm.ee> writes:
... the calendar says it's too late for 7.1.
Even for the _real_ bugfixes in gist.c ?
If he were submitting only bugfixes, we wouldn't be having this
discussion.
But he had very little incentive to fix bugs in the version he
would not use.
Look, I don't like postponing improvements either. But if we don't
adhere to project management discipline,
But should we do that _blindly_?
I'd think that improving/fixing things in seldom-visited corners of
postgres should be a little more tolerable than messing around in core.
we are never going to get
releases out the door at all --- or if we do, they'll be too buggy
to be reliable. It's not like "no new features during beta" is such
a draconian or difficult-to-understand rule.
I'd rather describe his changes as "a (bug)fix that required a major
rewrite" ;)
The RelFileNodeEquals() bug we found on Monday proves that no one had
yet done enough stress-testing on 7.1 to discover that multiple
databases were broken.
BTW, What do people use for stress-testing ?
Think about that for awhile before you campaign for inserting untested
new features at this point.
Rather new variants of little-tested features ;)
We need to focus on TESTING, people, not new features.
I make a personal promise to spend at least 5 hours of testing new GiST
functionality during this weekend if it is commited to 7.1 CVS.
(ok, I do it anyhow, just that currently I'm testing it using the
patches ;)
-------------------------
Hannu
On Thu, 11 Jan 2001, Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
I understand Tom's arguments and respect his experience, so I think it's
possible to put link to my page in 7.1 docs for people interested in
GiST features.Bear in mind that I only have one core vote ;-). We've already had some
private core discussion about whether to accept this patch now, and so
far I think I'm outvoted.
There are several Tom Lane, judge by your activity. You probably need
several votes.
Did I understand you to say that you'd added some regression tests for
GiST? That would lessen my unhappiness a little bit ...
Yes, we did. Currently all files are available from my page
http://www.sai.msu.su/~megera/postgres/gist/
I could submit them to hackers list if CORE people got consensus
Regards,
Oleg
regards, tom lane
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
On Thu, 11 Jan 2001, Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
I understand Tom's arguments and respect his experience, so I think it's
possible to put link to my page in 7.1 docs for people interested in
GiST features.Bear in mind that I only have one core vote ;-). We've already had some
private core discussion about whether to accept this patch now, and so
far I think I'm outvoted.There are several Tom Lane, judge by your activity. You probably need
several votes.Did I understand you to say that you'd added some regression tests for
GiST? That would lessen my unhappiness a little bit ...Yes, we did. Currently all files are available from my page
http://www.sai.msu.su/~megera/postgres/gist/
I could submit them to hackers list if CORE people got consensus
Okay, if there are appropriate regression tests, I'm going to say go for
it ...
Does anyone have any objections to my downloading the tar file (doing that
now), committing the changes and wrapping up a quick Beta4 just so that we
have a tar ball that is testable right away?
Save Lamar and the other packagers a bit of work by avoiding beta3
packages :)
just downloaded it and can't find any regression tests ... ?
On Thu, 11 Jan 2001, The Hermit Hacker wrote:
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
On Thu, 11 Jan 2001, Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
I understand Tom's arguments and respect his experience, so I think it's
possible to put link to my page in 7.1 docs for people interested in
GiST features.Bear in mind that I only have one core vote ;-). We've already had some
private core discussion about whether to accept this patch now, and so
far I think I'm outvoted.There are several Tom Lane, judge by your activity. You probably need
several votes.Did I understand you to say that you'd added some regression tests for
GiST? That would lessen my unhappiness a little bit ...Yes, we did. Currently all files are available from my page
http://www.sai.msu.su/~megera/postgres/gist/
I could submit them to hackers list if CORE people got consensusOkay, if there are appropriate regression tests, I'm going to say go for
it ...Does anyone have any objections to my downloading the tar file (doing that
now), committing the changes and wrapping up a quick Beta4 just so that we
have a tar ball that is testable right away?Save Lamar and the other packagers a bit of work by avoiding beta3
packages :)
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
The Hermit Hacker wrote:
Save Lamar and the other packagers a bit of work by avoiding beta3
packages :)
:-)
Well, working on those packages now. Fun stuff. I learn more about
more different parts of the code each new version, as the hang-ups
change from version to version. Although, thanks to Karl DeBisschop, I
now have a new secret weapon in my war Bwahaha... :-)
There have been more changes since 7.0 than in any previous version I
can remember as far as the build goes. Not a bad thing -- just a
learning experience.
At least the build won't change from beta3 to beta4.....
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
On Thu, 11 Jan 2001, The Hermit Hacker wrote:
just downloaded it and can't find any regression tests ... ?
it's in the contrib-intarray.tar.gz
gmake, gmake install, gmake installcheck
Oleg
On Thu, 11 Jan 2001, The Hermit Hacker wrote:
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
On Thu, 11 Jan 2001, Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
I understand Tom's arguments and respect his experience, so I think it's
possible to put link to my page in 7.1 docs for people interested in
GiST features.Bear in mind that I only have one core vote ;-). We've already had some
private core discussion about whether to accept this patch now, and so
far I think I'm outvoted.There are several Tom Lane, judge by your activity. You probably need
several votes.Did I understand you to say that you'd added some regression tests for
GiST? That would lessen my unhappiness a little bit ...Yes, we did. Currently all files are available from my page
http://www.sai.msu.su/~megera/postgres/gist/
I could submit them to hackers list if CORE people got consensusOkay, if there are appropriate regression tests, I'm going to say go for
it ...Does anyone have any objections to my downloading the tar file (doing that
now), committing the changes and wrapping up a quick Beta4 just so that we
have a tar ball that is testable right away?Save Lamar and the other packagers a bit of work by avoiding beta3
packages :)Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
On Thu, 11 Jan 2001, The Hermit Hacker wrote:
just downloaded it and can't find any regression tests ... ?
it's in the contrib-intarray.tar.gz
gmake, gmake install, gmake installcheck
erk, can we get this somehow done in such a way that its part of the
*standard* regression tests? so when ppl do 'make test', the GiST stuff
is checked also? My worry, as with others, isn't that GiST itself is
broken by the changes, its that *somehow* there is an interaction that is
with the rest of the system that isn't being tested ...
Oleg
On Thu, 11 Jan 2001, The Hermit Hacker wrote:
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
On Thu, 11 Jan 2001, Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
I understand Tom's arguments and respect his experience, so I think it's
possible to put link to my page in 7.1 docs for people interested in
GiST features.Bear in mind that I only have one core vote ;-). We've already had some
private core discussion about whether to accept this patch now, and so
far I think I'm outvoted.There are several Tom Lane, judge by your activity. You probably need
several votes.Did I understand you to say that you'd added some regression tests for
GiST? That would lessen my unhappiness a little bit ...Yes, we did. Currently all files are available from my page
http://www.sai.msu.su/~megera/postgres/gist/
I could submit them to hackers list if CORE people got consensusOkay, if there are appropriate regression tests, I'm going to say go for
it ...Does anyone have any objections to my downloading the tar file (doing that
now), committing the changes and wrapping up a quick Beta4 just so that we
have a tar ball that is testable right away?Save Lamar and the other packagers a bit of work by avoiding beta3
packages :)Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.orgRegards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
On Thu, 11 Jan 2001, The Hermit Hacker wrote:
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
On Thu, 11 Jan 2001, The Hermit Hacker wrote:
just downloaded it and can't find any regression tests ... ?
it's in the contrib-intarray.tar.gz
gmake, gmake install, gmake installcheckerk, can we get this somehow done in such a way that its part of the
*standard* regression tests? so when ppl do 'make test', the GiST stuff
is checked also? My worry, as with others, isn't that GiST itself is
broken by the changes, its that *somehow* there is an interaction that is
with the rest of the system that isn't being tested ...
No way, we need to load functions. there are several contributions
which depends on loaded functions. If you suggest how to do this
in general way, it would fine. To test GiST you need to define some
data structure ( in our case - RD-tree) and functions to access it
Oleg
On Thu, 11 Jan 2001, The Hermit Hacker wrote:
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
On Thu, 11 Jan 2001, Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
I understand Tom's arguments and respect his experience, so I think it's
possible to put link to my page in 7.1 docs for people interested in
GiST features.Bear in mind that I only have one core vote ;-). We've already had some
private core discussion about whether to accept this patch now, and so
far I think I'm outvoted.There are several Tom Lane, judge by your activity. You probably need
several votes.Did I understand you to say that you'd added some regression tests for
GiST? That would lessen my unhappiness a little bit ...Yes, we did. Currently all files are available from my page
http://www.sai.msu.su/~megera/postgres/gist/
I could submit them to hackers list if CORE people got consensusOkay, if there are appropriate regression tests, I'm going to say go for
it ...Does anyone have any objections to my downloading the tar file (doing that
now), committing the changes and wrapping up a quick Beta4 just so that we
have a tar ball that is testable right away?Save Lamar and the other packagers a bit of work by avoiding beta3
packages :)Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.orgRegards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
The Hermit Hacker <scrappy@hub.org> writes:
Does anyone have any objections to my downloading the tar file (doing that
now), committing the changes and wrapping up a quick Beta4 just so that we
have a tar ball that is testable right away?
I think we oughta review the changes at least a little bit before
pushing out a beta4... go ahead and commit though...
regards, tom lane
Lamar Owen <lamar.owen@wgcr.org> writes:
There have been more changes since 7.0 than in any previous version I
can remember as far as the build goes.
No surprise, considering the amount of work Peter E. has done on
cleaning up the configure, build, and install process --- exactly
the stuff that would affect RPM building. I trust you're finding
that the changes are for the better.
Unless Peter has plans he hasn't mentioned, future revs shouldn't
see so much activity there.
regards, tom lane
erk, can we get this somehow done in such a way that its part of the
*standard* regression tests? so when ppl do 'make test',
the GiST stuff is checked also? My worry, as with others, isn't that
GiST itself is broken by the changes, its that *somehow* there is an
interaction that is with the rest of the system that isn't being tested
...
No way, we need to load functions. there are several contributions
which depends on loaded functions. If you suggest how to do this
in general way, it would fine. To test GiST you need to define some
data structure ( in our case - RD-tree) and functions to access it
Look at regress/input/create_function_1.source for hints from
SPI tests...
Vadim
Import Notes
Resolved by subject fallback
On Thu, 11 Jan 2001, Mikheev, Vadim wrote:
erk, can we get this somehow done in such a way that its part of the
*standard* regression tests? so when ppl do 'make test',
the GiST stuff is checked also? My worry, as with others, isn't that
GiST itself is broken by the changes, its that *somehow* there is an
interaction that is with the rest of the system that isn't being tested...
No way, we need to load functions. there are several contributions
which depends on loaded functions. If you suggest how to do this
in general way, it would fine. To test GiST you need to define some
data structure ( in our case - RD-tree) and functions to access itLook at regress/input/create_function_1.source for hints from
SPI tests...
Thanks Vadim for tips. Will do this way, but tommorow. It's
3:19 am already and I have to sleep :-)
Vadim
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov <oleg@sai.msu.su> writes:
No way, we need to load functions. there are several contributions
which depends on loaded functions. If you suggest how to do this
in general way, it would fine. To test GiST you need to define some
data structure ( in our case - RD-tree) and functions to access itLook at regress/input/create_function_1.source for hints from
SPI tests...
Um, you do realize that a contrib module that gets used as part of the
regress tests may as well be mainstream? At least in terms of the
portability requirements it will have to meet?
I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?
regards, tom lane
Um, you do realize that a contrib module that gets used as part of the
regress tests may as well be mainstream? At least in terms of the
portability requirements it will have to meet?I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?
Ops, agreed.
And I fear that in current code there is no one GiST index
implementation -:( Should we worry about regress tests? -:)
Vadim
Import Notes
Resolved by subject fallback
Tom Lane wrote:
Um, you do realize that a contrib module that gets used as part of the
regress tests may as well be mainstream? At least in terms of the
portability requirements it will have to meet?
I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?
Last I checked, two contrib modules had to be built for regression
testing. But that was 7.0. (autoinc and refint.....).
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
Lamar Owen <lamar.owen@wgcr.org> writes:
I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?
Last I checked, two contrib modules had to be built for regression
testing.
Sure, but they've been there awhile. All of my concerns here are
schedule-driven: do we really want to be wringing out a new contrib
module, to the point where it will run everywhere, before we can
release 7.1?
regards, tom lane
Tom Lane wrote:
Lamar Owen <lamar.owen@wgcr.org> writes:
I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?
Last I checked, two contrib modules had to be built for regression
testing.
Sure, but they've been there awhile. All of my concerns here are
schedule-driven: do we really want to be wringing out a new contrib
module, to the point where it will run everywhere, before we can
release 7.1?
Are the benefits worth the effort? Can the current GiST developers pull
it off in time?
If the answer to either question is not a resounding YES then we really
don't need to go down this road. Either leave it in contrib and
regression testless (with a test script in the contrib), or make it a
feature patch.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
On Thu, 11 Jan 2001, Tom Lane wrote:
Lamar Owen <lamar.owen@wgcr.org> writes:
I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?Last I checked, two contrib modules had to be built for regression
testing.Sure, but they've been there awhile. All of my concerns here are
schedule-driven: do we really want to be wringing out a new contrib
module, to the point where it will run everywhere, before we can
release 7.1?
Hrmmm ... just a thought here, but how about a potential 'interactive'
regression test, where it asks if you want to run regress on GiST? If so,
do it, if not, ignore it ... ?
On Thu, 11 Jan 2001, Mikheev, Vadim wrote:
Um, you do realize that a contrib module that gets used as part of the
regress tests may as well be mainstream? At least in terms of the
portability requirements it will have to meet?I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?Ops, agreed.
And I fear that in current code there is no one GiST index
implementation -:( Should we worry about regress tests? -:)
Yes, we had to write contrib module even to test GiST. People,
I'm really confused after reading all of messages.
GiST is just an interface and to test any interface you need 2 sides.
In current code there is only one side. old GiST code live
untested for years. What's the problem ? It's the problem of
current regression test, mostly.
Ok. We could rewrite R-Tree to use GiST and make regression test which
will not make people nervous. But this certainly not for 7.1 and most
probable without us. Author of R-Tree could write this easily.
I read Bruce's interview and was really relaxed -
how everything is going well. Bruce, we need your opinion.
Oleg
Vadim
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
OK. We found an old implementation of R-Tre using GiST (Pg95)
and we'll try to implement regression test using R-Tree
it's anyway will be a good test.
Regards,
Oleg
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
On Thu, 11 Jan 2001, Mikheev, Vadim wrote:
Um, you do realize that a contrib module that gets used as part of the
regress tests may as well be mainstream? At least in terms of the
portability requirements it will have to meet?I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?Ops, agreed.
And I fear that in current code there is no one GiST index
implementation -:( Should we worry about regress tests? -:)Yes, we had to write contrib module even to test GiST. People,
I'm really confused after reading all of messages.
GiST is just an interface and to test any interface you need 2 sides.
In current code there is only one side. old GiST code live
untested for years. What's the problem ? It's the problem of
current regression test, mostly.
Ok. We could rewrite R-Tree to use GiST and make regression test which
will not make people nervous. But this certainly not for 7.1 and most
probable without us. Author of R-Tree could write this easily.
I read Bruce's interview and was really relaxed -
how everything is going well. Bruce, we need your opinion.Oleg
Vadim
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
On Thu, 11 Jan 2001, Mikheev, Vadim wrote:
Um, you do realize that a contrib module that gets used as part of the
regress tests may as well be mainstream? At least in terms of the
portability requirements it will have to meet?I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?Ops, agreed.
And I fear that in current code there is no one GiST index
implementation -:( Should we worry about regress tests? -:)Yes, we had to write contrib module even to test GiST. People,
I'm really confused after reading all of messages.
GiST is just an interface and to test any interface you need 2 sides.
In current code there is only one side. old GiST code live
untested for years. What's the problem ? It's the problem of
current regression test, mostly.
Ok. We could rewrite R-Tree to use GiST and make regression test which
will not make people nervous. But this certainly not for 7.1 and most
probable without us. Author of R-Tree could write this easily.
The problem is that there is inadequate amount of time to fully test the
the regression tests among all of the platforms that we support ...
therefore, we can't test GiST among all of the platforms we support ...
for instance, can you guarantee that _int.c will compile on *every*
platform that PostgreSQL supports? That it will operate properly? Have
you tested that?
On Fri, 12 Jan 2001, Hannu Krosing wrote:
Oleg Bartunov wrote:
OK. We found an old implementation of R-Tre using GiST (Pg95)
and we'll try to implement regression test using R-Tree
it's anyway will be a good test.How is it different than using RD-tree for tests ?
No difference at all ! It's just another implemetation of R-Tree.
Can you do it usin already compiled-in functions and modifying
things only at SQL level ?
unfortunately not ! Current postgres code has nothing connected with
GiST and this is a problem ! How to test interface code without
having two sides ? I understand we don't want to have another reason
for complaints about non-working regression test. I never got
regression test passed 100% on my Linux box with almost all versions
of PostgreSQL but I could live with that. What's wrong with
warning message if GiST test not passed ?
Or is it just much simpler ?
I'm interesting to test performance of built-in R-Tree and R-Tree + GiST.
Oleg
---------------------
Hannu
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Import Notes
Reply to msg id not found: 3A5F2B45.1EC448AA@tm.ee | Resolved by subject fallback
Oleg Bartunov <oleg@sai.msu.su> writes:
What's wrong with
warning message if GiST test not passed ?
You're being *way* too optimistic. An output discrepancy in a test of
GIST we could live with. But think about other scenarios:
1. GIST test coredumps on some platforms. This corrupts other tests
(at least through the "system is starting up" failure mode), thus
masking problems that we actually care about.
2. GIST test code does not compile on some platforms, causing "make check"
to fail completely.
At this point my vote is to leave the GIST test in contrib for 7.1.
Anyone who actually cares about GIST (to be blunt: all three of you)
can run it as a separate step. I don't want it in the standard regress
tests until 7.2, when we will have a reasonable amount of time to test
and debug the test.
regards, tom lane
IMHO, giving out real test results, even negative, instead of leaving
things untested would be a honest thing to do.
afaict there are several concerns or decisions, and we've made a few
already:
Re: gist.c patches...
1) Oleg and Hannu are committed to testing the repaired GiST as soon as
it is in the main tree. They are both testing already with the patched
version.
2) They will try to contact Gene to encourage testing with Gene's
application, though they have no reason to suspect from their own
testing that Gene's stuff will break.
3) There is a consensus that the gist.c patches should appear in the 7.1
release, to allow useful work with GiST and to enable further
development. So it is OK to commit the gist.c patches based on Oleg's
and Hannu's existing and future test plan.
Re: regression tests...
4) We all would like to see some regression tests of GiST. Tom Lane has
(rightly) expressed concern over unforeseen breakage in the regression
flow when done on other platforms.
5) Oleg has some regression-capable test code available for contrib, but
has indicated that fully (re)writing the regression tests will take too
much time.
6) We have at least two committed testers for the 7.1 release for the
GiST features. That is two more than we've ever had before (afacr Gene
didn't participate in the end-stage beta cycle, but I may not be
remembering correctly) so the risks that something is not right are
greatly reduced, to below the risks of same on the day of release for
previous versions.
8) Additional regression testing is required asap, but may not be
allowed into the default 7.1 test sequence.
How about adding an optional test a la "bigtest" for GiST for this
release? It could go mainstream for 7.1.x or for 7.2 as we get more
experience with it. This is just a suggestion and I'm sure there are
other possibilities. I'm pretty sure we agree on most of points 1-8, and
that 1-3 are resolved. Comments?
- Thomas
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
On Fri, 12 Jan 2001, Hannu Krosing wrote:
Oleg Bartunov wrote:
OK. We found an old implementation of R-Tre using GiST (Pg95)
and we'll try to implement regression test using R-Tree
it's anyway will be a good test.How is it different than using RD-tree for tests ?
No difference at all ! It's just another implemetation of R-Tree.
Can you do it usin already compiled-in functions and modifying
things only at SQL level ?unfortunately not ! Current postgres code has nothing connected with
GiST and this is a problem ! How to test interface code without
having two sides ? I understand we don't want to have another reason
for complaints about non-working regression test. I never got
regression test passed 100% on my Linux box with almost all versions
of PostgreSQL but I could live with that. What's wrong with
warning message if GiST test not passed ?
It has *nothing* to do with passing or not, it has to do with timing of
hte patches ... had they come in before we went beta, this would all have
been a no-brainer ... because they didn't, the problem arises ...
GiST changes are included ... testing of GiST changes aren't integrated
... can we *please* drop this whole thing already, as its really
detracting from getting *real* work done with very little, to no, benefit
...
On Fri, 12 Jan 2001, Tom Lane wrote:
Oleg Bartunov <oleg@sai.msu.su> writes:
What's wrong with
warning message if GiST test not passed ?You're being *way* too optimistic. An output discrepancy in a test of
GIST we could live with. But think about other scenarios:1. GIST test coredumps on some platforms. This corrupts other tests
(at least through the "system is starting up" failure mode), thus
masking problems that we actually care about.2. GIST test code does not compile on some platforms, causing "make check"
to fail completely.At this point my vote is to leave the GIST test in contrib for 7.1.
Anyone who actually cares about GIST (to be blunt: all three of you)
can run it as a separate step. I don't want it in the standard regress
tests until 7.2, when we will have a reasonable amount of time to test
and debug the test.
Agreed ... now let's move onto more important things, cause we've spent
much too long on this as it is ...
Namely, should we bundle up a beta4 this weeekend, so that the GiST
changes are in place for further testing, or hold off for ... ?
On Fri, 12 Jan 2001, Thomas Lockhart wrote:
How about adding an optional test a la "bigtest" for GiST for this
release? It could go mainstream for 7.1.x or for 7.2 as we get more
experience with it. This is just a suggestion and I'm sure there are
other possibilities. I'm pretty sure we agree on most of points 1-8, and
that 1-3 are resolved. Comments?
make GIST_TEST=yes
to include GiST testing would be cool, if it can be done ... this way
Tom's worry about non-GiST users having bad regress tests is appeased ...
but I do agree with Tom that mainstreaming the GiST testing would be a bad
idea ... if we could somehow include it as an optional test (as you say,
ala bigtest), then, if nothing else, it saves having to cd to the contrib
directory and run it there ... ala one stop shopping ...
*But*, for the 3 ppl we've pointed out as users of GiST, this is
definitely not a priority issue ... if we can do it, great, if not, no
sweat either ...
Thomas Lockhart <lockhart@alumni.caltech.edu> writes:
How about adding an optional test a la "bigtest" for GiST for this
release?
We could do that, but it seems like rather pointless effort, compared
to just telling people "go run the tests in these contrib modules if
you want to test GIST".
I have no objection to fully integrating some GIST test(s) for 7.2.
I just don't want to deal with it at this late stage of the 7.1 cycle.
We have a long list of considerably more mainstream to-do items yet
to deal with ...
regards, tom lane
The Hermit Hacker <scrappy@hub.org> writes:
Namely, should we bundle up a beta4 this weeekend, so that the GiST
changes are in place for further testing, or hold off for ... ?
First I'd like to finish a couple of open items I have, like fixing
the CRIT_SECTION code so that SIGTERM response will not occur when
we are holding a spinlock. Should be able to get this stuff done in
a day or two, if I quit arguing about GIST and get back to work...
regards, tom lane
Tom Lane wrote:
Um, you do realize that a contrib module that gets used as part of the
regress tests may as well be mainstream? At least in terms of the
portability requirements it will have to meet?
_If_ we want to have a tested GiST (and not the "probably works but
really has some nasty known bugs" one) we need to write _tests_.
To test it we need something that makes use of it.
As the only things that make use of it are extensions we need to
make use of them in tests.
So I propose the following :
1. Keep the fixed (new) gist.c in the main codebase
2. make use of the RD-index and/or Gene's tests in contrib in regression
tests
3. Tellpeople beforehand that it is not the end of the world
if GiST _tests_ fail on their platform
I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?
There can be always "expected" discrepancies in regress tests, and
failing GiST test just tells people that if they want to use GiST on
their platform they must probably fix things in core code as well.
Currently they have to find it out the hard way - first lot of work
trying to "fix" their own code and only then the bright idea that
maybe it is actually broken in the core.
IMHO, giving out real test results, even negative, instead of leaving
things untested would be a honest thing to do.
-----------------
Hannu
Oleg Bartunov wrote:
OK. We found an old implementation of R-Tre using GiST (Pg95)
and we'll try to implement regression test using R-Tree
it's anyway will be a good test.
How is it different than using RD-tree for tests ?
Can you do it usin already compiled-in functions and modifying
things only at SQL level ?
Or is it just much simpler ?
---------------------
Hannu
Thomas Lockhart writes:
How about adding an optional test a la "bigtest" for GiST for this
release?
An optional test is like no test at all. No one runs optional tests. If
the test is supposed to work then it should be mainstream. If the test
might not work then you better go back and figure out what you're testing.
If the test might not *compile* (which is probably the more severe problem
that people are concerned about) then this idea won't help that at all
unless you want to rework the regression test driver framework as well.
--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/
An optional test is like no test at all. No one runs optional tests. If
the test is supposed to work then it should be mainstream. If the test
might not work then you better go back and figure out what you're testing.
If the test might not *compile* (which is probably the more severe problem
that people are concerned about) then this idea won't help that at all
unless you want to rework the regression test driver framework as well.
I agree completely. This is just a transition phase to get GiST into the
mainstream.
- Thomas
At this point my vote is to leave the GIST test in contrib for 7.1.
Anyone who actually cares about GIST (to be blunt: all three of you)
can run it as a separate step. I don't want it in the standard regress
tests until 7.2, when we will have a reasonable amount of time to test
and debug the test.
Agreed. I want the GIST fixes in 7.1, but adding a new test at this
point is too risky.
The issue is that only the GIST people will be using the GIST fixes,
while adding it to the regression test will affect all users, which is
too risky at this point.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026Agreed ... now let's move onto more important things, cause we've spent
much too long on this as it is ...Namely, should we bundle up a beta4 this weeekend, so that the GiST
changes are in place for further testing, or hold off for ... ?
I would hold off. GIST people can download the snapshot. Others aren't
interested in GIST.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026On Fri, 12 Jan 2001, Tom Lane wrote:
Thomas Lockhart <lockhart@alumni.caltech.edu> writes:
How about adding an optional test a la "bigtest" for GiST for this
release?We could do that, but it seems like rather pointless effort, compared
to just telling people "go run the tests in these contrib modules if
you want to test GIST".I have no objection to fully integrating some GIST test(s) for 7.2.
I just don't want to deal with it at this late stage of the 7.1 cycle.
We have a long list of considerably more mainstream to-do items yet
to deal with ...
Not up to us to deal with, its up to Oleg ...
Oleg, if you could work on and submit patches for this before the release,
that would be appreciated ... it might also serve to increase visibility
of GiST if ppl know there is a regress test for it ...
On Fri, 12 Jan 2001, Tom Lane wrote:
The Hermit Hacker <scrappy@hub.org> writes:
Namely, should we bundle up a beta4 this weeekend, so that the GiST
changes are in place for further testing, or hold off for ... ?First I'd like to finish a couple of open items I have, like fixing
the CRIT_SECTION code so that SIGTERM response will not occur when
we are holding a spinlock. Should be able to get this stuff done in
a day or two, if I quit arguing about GIST and get back to work...
Okay, let's scheduale for Monday then if we can ... unless someone comes
across something major like we did with the whole beta2/beta3 release :)
I am sorry I wasn't listening -- I may have helped by at least
answering the direct questions and by testing. I have, in fact,
positively tested both my and Oleg's code in the today's snapshot on a
number of linux and FreeBSD systems. I failed on this one:
SunOS typhoon 5.7 Generic_106541-10 sun4u sparc SUNW,Ultra-1
on which configure didn't detect the absence of libz.so
I don't think my applications are affected by Oleg's changes. But I
understand the tension that occurred during the past few days and even
though I am now satisfied with the agreement you seem to have
achieved, I could have hardly influenced it in any reasonable way. I
am as sympathetic with the need for a smooth an solid code control as
I am with promoting great features (or, in this case, just keeping a
feature alive). So, if I were around at the time I was asked to vote,
I wouldn't know how. I usually find it difficult to take sides in
"Motherhood vs. Clean Air" debates. It is true that throwing a core
during a regression test does gives one a black eye. It is also true
that there are probably hundreds of possible users, ignorant of the
GiST, trying to invent surrogate solutions. As far as I am concerned,
I will be satisfied with whatever solution you arrive at. I am pleased
that in this neighborhood, reason prevails over faith.
--Gene
selkovjr@mcs.anl.gov writes:
I am sorry I wasn't listening -- I may have helped by at least
answering the direct questions and by testing. I have, in fact,
positively tested both my and Oleg's code in the today's snapshot on a
number of linux and FreeBSD systems. I failed on this one:
SunOS typhoon 5.7 Generic_106541-10 sun4u sparc SUNW,Ultra-1
on which configure didn't detect the absence of libz.so
Really? Details please. It's hard to see how it could have messed
up on that.
regards, tom lane
Hi,
I've put R-Tree realization using GiST (yet another test of our changes in
gist code )on my gist page http://www.sai.msu.su/~megera/postgres/gist/
Also, I've put some GiST related papers for interested readers.
The package( contrib-rtree_box_gist.tar.gz ) is built for 7.1.
If you find it's interesting you may include it into contrib area for 7.1
from README.rtree_box_gist:
1. One interesting thing is that insertion time for built-in R-Tree is
about 8 times more than ones for GiST implementation of R-Tree !!!
2. Postmaster requires much more memory for built-in R-Tree
3. Search time depends on dataset. In our case we got:
+------------+-----------+--------------+
|Number boxes|R-tree, sec|R-tree using |
| | | GiST, sec |
+------------+-----------+--------------+
| 10| 0.002| 0.002|
+------------+-----------+--------------+
| 100| 0.002| 0.002|
+------------+-----------+--------------+
| 1000| 0.002| 0.002|
+------------+-----------+--------------+
| 10000| 0.015| 0.025|
+------------+-----------+--------------+
| 20000| 0.029| 0.048|
+------------+-----------+--------------+
| 40000| 0.055| 0.092|
+------------+-----------+--------------+
| 80000| 0.113| 0.178|
+------------+-----------+--------------+
| 160000| 0.338| 0.337|
+------------+-----------+--------------+
| 320000| 0.674| 0.673|
+------------+-----------+--------------+ Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
selkovjr@mcs.anl.gov writes:
on which configure didn't detect the absence of libz.so
Really? Details please. It's hard to see how it could have messed
up on that.
I didn't look well enough -- I apologize. The library is there, but
ld.so believes it is not:
typhoon> postmaster
ld.so.1: postmaster: fatal: libz.so: open failed: No such file or directory
Killed
Odd. Can you show us the part of config.log that relates to zlib?
It's strange that configure's check to see if zlib is linkable should
succeed, only to have the live startup fail. Is it possible that
you ran configure with a different library search path (LD_LIBRARY_PATH
or local equivalent) than you are using now?
It's suspicious that the error message mentions libz.so when the actual
file name is libz.so.1, but I still don't see how that could result in
configure's link test succeeding but the executable not running.
regards, tom lane
Import Notes
Reply to msg id not found: 200101150639.AAA14799@selkovjr.xnet.com
Tom Lane writes:
selkovjr@mcs.anl.gov writes:
...
SunOS typhoon 5.7 Generic_106541-10 sun4u sparc SUNW,Ultra-1
on which configure didn't detect the absence of libz.so
Really? Details please. It's hard to see how it could have messed
up on that.
Tom,
I didn't look well enough -- I apologize. The library is there, but
ld.so believes it is not:
typhoon> postmaster
ld.so.1: postmaster: fatal: libz.so: open failed: No such file or directory
Killed
This may very well be just my ISP's problem.
Anyway, the details are:
1. My (relevant) environment:
LD_LIBRARY_PATH=/usr/openwin/lib:/usr/lib:/usr/ucblib:/usr/ccs/lib
PGLIB=/home/customer/selkovjr/pgsql/lib
PGDATA=/home/customer/selkovjr/pgsql/data
PATH=/usr/local/vendor/SUNWspro/bin:/usr/local/bin:/usr/local/gnu/bin:/usr/local/GNU/bin:/usr/sbin:/usr/bin:/usr/ccs/bin:/usr/ucb:/etc:/usr/etc:/usr/openwin/bin:/home/customer/selkovjr/bin:./usr/local/bin::/home/customer/selkovjr/pgsql/bin
2. I built postgres (from the snapshot of Jan 13) with:
./configure --prefix=/home/customer/selkovjr/pgsql
make
make install
3. initdb worked.
4. The library in question is in /usr/openwin/lib:
typhoon> ls -l /usr/openwin/lib | grep libz
-rwxr-xr-x 1 root bin 97836 Sep 23 1999 libz.a
-rwxr-xr-x 1 root bin 70452 Sep 23 1999 libz.so.1
I can't think of anything else. Is there a one-liner to test libz? I
believe I have successfully tested and run 6.5.3 in the same
environment.
--Gene
Tom Lane writes:
selkovjr@mcs.anl.gov writes:
on which configure didn't detect the absence of libz.so
Really? Details please. It's hard to see how it could have messed
up on that.I didn't look well enough -- I apologize. The library is there, but
ld.so believes it is not:typhoon> postmaster
ld.so.1: postmaster: fatal: libz.so: open failed: No such file or directory
KilledOdd. Can you show us the part of config.log that relates to zlib?
configure:4179: checking for zlib.h
configure:4189: gcc -E conftest.c >/dev/null 2>conftest.out
configure:4207: checking for inflate in -lz
configure:4226: gcc -o conftest conftest.c -lz -lgen -lnsl -lsocket -ldl -lm -lreadline -ltermcap -lcurses 1>&5
configure:4660: checking for crypt.h
This doesn't tell me much. But I modified configure to exit right
after this, without removing conftest*, and when I ran conftest it came
back with the same message:
typhoon> ./conftest
ld.so.1: ./conftest: fatal: libz.so: open failed: No such file or directory
Killed
It's strange that configure's check to see if zlib is linkable should
succeed, only to have the live startup fail.
It is. In this line:
if { (eval echo configure:4226: \"$ac_link\") 1>&5; (eval $ac_link) 2>&5; } && test -s conftest${ac_exeext}; then
why is conftest tested for size instead of being executed?
Is it possible that
you ran configure with a different library search path (LD_LIBRARY_PATH
or local equivalent) than you are using now?
No, I didn't alter it. I am using the system-wide settings.
It's suspicious that the error message mentions libz.so when the actual
file name is libz.so.1, but I still don't see how that could result in
configure's link test succeeding but the executable not running.
That puzzles me as well. It seems to be because there is no libz.so on
the system. For if I do this:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/customer/selkovjr/lib
ln -s /usr/openwin/lib/libz.so.1 ~/lib/libz.so
the libz problem is gone, only to be followed by the next one:
typhoon> ./conftest
ld.so.1: ./conftest: fatal: libreadline.so: open failed: No such file or directory
The odd thing is, there is no libreadline.so* on this system. Here's the corresponding part of config.log:
configure:3287: checking for library containing readline
configure:3305: gcc -o conftest conftest.c -ltermcap -lcurses 1>&5
Undefined first referenced
symbol in file
readline /var/tmp/ccxxiW3R.o
ld: fatal: Symbol referencing errors. No output written to conftest
collect2: ld returned 1 exit status
configure: failed program was:
#line 3294 "configure"
#include "confdefs.h"
/* Override any gcc2 internal prototype to avoid an error. */
/* We use char because int might match the return type of a gcc2
builtin and then its argument prototype would still apply. */
char readline();
int main() {
readline()
; return 0; }
configure:3327: gcc -o conftest conftest.c -lreadline -ltermcap -lcurses
1>&5
This system is probaly badly misconfigured, but it would be great if
configure could see that. By the way, would you mind if I asked you to
log in and take a look? Is there a phone number where I can get you
with the password? I am not sure whether such tests could be of any
value, but it's the only Sun machine available to me for testing.
Thank you,
--Gene
selkovjr@mcs.anl.gov writes:
configure:4207: checking for inflate in -lz
configure:4226: gcc -o conftest conftest.c -lz -lgen -lnsl -lsocket -ldl -lm -lreadline -ltermcap -lcurses 1>&5
configure:4660: checking for crypt.h
This doesn't tell me much. But I modified configure to exit right
after this, without removing conftest*, and when I ran conftest it came
back with the same message:
typhoon> ./conftest
ld.so.1: ./conftest: fatal: libz.so: open failed: No such file or directory
Killed
It's strange that configure's check to see if zlib is linkable should
succeed, only to have the live startup fail.
This system is probaly badly misconfigured, but it would be great if
configure could see that.
Gene and I looked into this, and the cause of the misbehavior is this:
gcc on this installation is set to search /usr/local/lib (along with the
usual system library directories). libz.so and libreadline.so are
indeed in /usr/local/lib, so configure's tests to see if they can be
linked against will succeed. But he had LD_LIBRARY_PATH set to a list
that did *not* include /usr/local/lib, so actually firing up the
executable would fail.
As he says, it'd be nice if configure could either prevent this or at
least detect it. Not sure about a good way to do that --- any ideas?
regards, tom lane
Tom Lane writes:
Gene and I looked into this, and the cause of the misbehavior is this:
gcc on this installation is set to search /usr/local/lib (along with the
usual system library directories). libz.so and libreadline.so are
indeed in /usr/local/lib, so configure's tests to see if they can be
linked against will succeed. But he had LD_LIBRARY_PATH set to a list
that did *not* include /usr/local/lib, so actually firing up the
executable would fail.
You get what you pay for. If you're running executables from configure
you're asking for it.
This setup is a poor man's cross-compilation situation because the system
you're compiling on is not identically configured to the system you're
going to run on. (Strictly speaking, the behaviour of a test program
might even vary with different LD_LIBRARY_PATH settings.)
So
a) PostgreSQL does not support cross-compilation (yet). Too bad.
b) We could get rid of all executition time checks in configure (to
remedy (a)). This is one of my plans for the future.
c) You could move the execution time checks up before the suspicious
library checks, but I'm afraid that this will only cure a particular
symptom and might introduce other problems.
I'd say, you're stuck.
--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/
On Fri, Jan 19, 2001 at 12:46:53AM +0100, Peter Eisentraut wrote:
Tom Lane writes:
Gene and I looked into this, and the cause of the misbehavior is this:
gcc on this installation is set to search /usr/local/lib (along with the
usual system library directories). libz.so and libreadline.so are
indeed in /usr/local/lib, so configure's tests to see if they can be
linked against will succeed. But he had LD_LIBRARY_PATH set to a list
that did *not* include /usr/local/lib, so actually firing up the
executable would fail.You get what you pay for. If you're running executables from configure
you're asking for it.This setup is a poor man's cross-compilation situation because the system
you're compiling on is not identically configured to the system you're
going to run on. (Strictly speaking, the behaviour of a test program
might even vary with different LD_LIBRARY_PATH settings.)So
a) PostgreSQL does not support cross-compilation (yet). Too bad.
b) We could get rid of all executition time checks in configure (to
remedy (a)). This is one of my plans for the future.c) You could move the execution time checks up before the suspicious
library checks, but I'm afraid that this will only cure a particular
symptom and might introduce other problems.I'd say, you're stuck.
--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/
Wouldn't a -Wl,-R/usr/local/lib have helped?
Cheers,
Patrick
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
Wouldn't a -Wl,-R/usr/local/lib have helped?
Well, yeah, but how would we know to do that? The fact that gcc is
searching /usr/local/lib is completely unknown to configure.
regards, tom lane
I have added the URL to the GIST SGML docs.
Hi,
I've put R-Tree realization using GiST (yet another test of our changes in
gist code )on my gist page http://www.sai.msu.su/~megera/postgres/gist/
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Also, I've put some GiST related papers for interested readers.
The package( contrib-rtree_box_gist.tar.gz ) is built for 7.1.
If you find it's interesting you may include it into contrib area for 7.1from README.rtree_box_gist:
1. One interesting thing is that insertion time for built-in R-Tree is about 8 times more than ones for GiST implementation of R-Tree !!! 2. Postmaster requires much more memory for built-in R-Tree 3. Search time depends on dataset. In our case we got: +------------+-----------+--------------+ |Number boxes|R-tree, sec|R-tree using | | | | GiST, sec | +------------+-----------+--------------+ | 10| 0.002| 0.002| +------------+-----------+--------------+ | 100| 0.002| 0.002| +------------+-----------+--------------+ | 1000| 0.002| 0.002| +------------+-----------+--------------+ | 10000| 0.015| 0.025| +------------+-----------+--------------+ | 20000| 0.029| 0.048| +------------+-----------+--------------+ | 40000| 0.055| 0.092| +------------+-----------+--------------+ | 80000| 0.113| 0.178| +------------+-----------+--------------+ | 160000| 0.338| 0.337| +------------+-----------+--------------+ | 320000| 0.674| 0.673| +------------+-----------+--------------+Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026