index fix report
I am still waiting for the first report from my index fix.
Anyone?
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)
I am still waiting for the first report from my index fix.
Anyone?
OK, here is my interim report: everything is broken :)
Looks like David's patches busted several Makefiles. Will poke at it and
let you know.
In the meantime, I can't test your fixes; maybe next time wait a few
hours to let things settle down before adding new breakage? *wink*
- Tom
gcc -I../../../include -I../../../backend -O2 -m486 -Wall
-Wmissing-prototypes -I../.. -Wno-error -c geqo_ox2.c -o geqo_ox2.o
ld -r -o SUBSYS.o geqo_copy.o geqo_eval.o geqo_main.o geqo_misc.o
geqo_params.o geqo_paths.o geqo_pool.o geqo_recombination.o
geqo_selection.o geqo_erx.o geqo_pmx.o geqo_cx.o geqo_px.o geqo_ox1.o
geqo_ox2.o
make[3]: Leaving directory
`/opt/postgres/pgsql/src/backend/optimizer/geqo'
for i in path plan prep util geqo; do make -C $i prep/SUBSYS.o; done
make[3]: Entering directory
`/opt/postgres/pgsql/src/backend/optimizer/path'
make[3]: *** No rule to make target `prep/SUBSYS.o'. Stop.
make[3]: Leaving directory
`/opt/postgres/pgsql/src/backend/optimizer/path'
make[3]: Entering directory
`/opt/postgres/pgsql/src/backend/optimizer/plan'
make[3]: *** No rule to make target `prep/SUBSYS.o'. Stop.
make[3]: Leaving directory
`/opt/postgres/pgsql/src/backend/optimizer/plan'
make[3]: Entering directory
`/opt/postgres/pgsql/src/backend/optimizer/prep'
make[3]: *** No rule to make target `prep/SUBSYS.o'. Stop.
make[3]: Leaving directory
`/opt/postgres/pgsql/src/backend/optimizer/prep'
make[3]: Entering directory
`/opt/postgres/pgsql/src/backend/optimizer/util'
make[3]: *** No rule to make target `prep/SUBSYS.o'. Stop.
make[3]: Leaving directory
`/opt/postgres/pgsql/src/backend/optimizer/util'
make[3]: Entering directory
`/opt/postgres/pgsql/src/backend/optimizer/geqo'
make[3]: *** No rule to make target `prep/SUBSYS.o'. Stop.
make[3]: Leaving directory
`/opt/postgres/pgsql/src/backend/optimizer/geqo'
make[2]: *** [prep/SUBSYS.o] Error 2
make[2]: Leaving directory `/opt/postgres/pgsql/src/backend/optimizer'
make[1]: *** [optimizer.dir] Error 2
make[1]: Leaving directory `/opt/postgres/pgsql/src/backend'
make: *** [install] Error 2
I am still waiting for the first report from my index fix.
Anyone?OK, here is my interim report: everything is broken :)
Looks like David's patches busted several Makefiles. Will poke at it and
let you know.In the meantime, I can't test your fixes; maybe next time wait a few
hours to let things settle down before adding new breakage? *wink*
It is compiling here now. Strange. There are few problems with
libpgtcl and psql.c that I am fixing now.
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)OK, it looks like there is a missing file in the tree. I had to modify
./backend/optimizer/prep/Makefile
to remove a reference to a nonexistant object file prepkeyset.o
presumably needing to be built from prepkeyset.c.That got things building until the next link step, when some routines
were a missin'. Got that file anywhere? Seems useful...
No wonder it compiles here so nicely. Adding it now. I will also do a
make distclean and cvs to see if any other files are missing.
- Tom
commands/SUBSYS.o: In function `parse_ksqo':
commands/SUBSYS.o(.text+0xdff4): undefined reference to
`_use_keyset_query_optimizer'
commands/SUBSYS.o(.text+0xe014): undefined reference to
`_use_keyset_query_optimizer'
commands/SUBSYS.o: In function `show_ksqo':
commands/SUBSYS.o(.text+0xe045): undefined reference to
`_use_keyset_query_optimizer'
commands/SUBSYS.o: In function `reset_ksqo':
commands/SUBSYS.o(.text+0xe085): undefined reference to
`_use_keyset_query_optimizer'
optimizer/SUBSYS.o: In function `planner':
optimizer/SUBSYS.o(.text+0x564b): undefined reference to
`transformKeySetQuery'
make[1]: *** [postgres] Error 1
make[1]: Leaving directory `/opt/postgres/pgsql/src/backend'
make: *** [install] Error 2
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)Import Notes
Reply to msg id not found: 35EE2504.F30AEB30@alumni.caltech.edu | Resolved by subject fallback
OK, it looks like there is a missing file in the tree. I had to modify
./backend/optimizer/prep/Makefile
to remove a reference to a nonexistant object file prepkeyset.o
presumably needing to be built from prepkeyset.c.
That got things building until the next link step, when some routines
were a missin'. Got that file anywhere? Seems useful...
- Tom
commands/SUBSYS.o: In function `parse_ksqo':
commands/SUBSYS.o(.text+0xdff4): undefined reference to
`_use_keyset_query_optimizer'
commands/SUBSYS.o(.text+0xe014): undefined reference to
`_use_keyset_query_optimizer'
commands/SUBSYS.o: In function `show_ksqo':
commands/SUBSYS.o(.text+0xe045): undefined reference to
`_use_keyset_query_optimizer'
commands/SUBSYS.o: In function `reset_ksqo':
commands/SUBSYS.o(.text+0xe085): undefined reference to
`_use_keyset_query_optimizer'
optimizer/SUBSYS.o: In function `planner':
optimizer/SUBSYS.o(.text+0x564b): undefined reference to
`transformKeySetQuery'
make[1]: *** [postgres] Error 1
make[1]: Leaving directory `/opt/postgres/pgsql/src/backend'
make: *** [install] Error 2
I am still waiting for the first report from my index fix.
Anyone?OK, here is my interim report: everything is broken :)
Looks like David's patches busted several Makefiles. Will poke at it and
let you know.In the meantime, I can't test your fixes; maybe next time wait a few
hours to let things settle down before adding new breakage? *wink*
OK, everything is there now.
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)make distclean and cvs to see if any other files are missing.
More breakage. Will disable the tcl stuff and see if I can proceed...
- Tom
gcc -I../../include -I../../backend -O2 -m486 -Wall
-Wmissing-prototypes -I../../backend -I../../include
-I../../interfaces/libpq -fpic -c pgtclCmds.c -o pgtclCmds.o
pgtclCmds.c: In function `Pg_connect':
pgtclCmds.c:366: parse error before `else'
pgtclCmds.c:373: warning: control reaches end of non-void function
pgtclCmds.c: In function `Pg_listen':
pgtclCmds.c:1460: parse error at end of input
make[2]: *** [pgtclCmds.o] Error 1
make[2]: Leaving directory `/opt/postgres/pgsql/src/interfaces/libpgtcl'
make[1]: *** [install] Error 2
make[1]: Leaving directory `/opt/postgres/pgsql/src/interfaces'
make: *** [install] Error 2
make distclean and cvs to see if any other files are missing.
... And more breakage ... Fortunately I had a psql from yesterday lying
around, so will continue testing ...
- Tom
psql.c: In function `SendQuery':
psql.c:1139: warning: passing arg 1 of `handleCopyIn' from incompatible
pointer type
psql.c:1143: warning: passing arg 1 of `handleCopyIn' from incompatible
pointer type
psql.c: At top level:
psql.c:2957: conflicting types for `handleCopyIn'
psql.c:153: previous declaration of `handleCopyIn'
make[2]: *** [psql.o] Error 1
make[2]: Leaving directory `/opt/postgres/pgsql/src/bin/psql'
make[2]: Entering directory `/opt/postgres/pgsql/src/bin/pg_dump'
OK, everything is there now.
The select_having test now passes. The other tests which were broken
yesterday are still broken:
constraints .. failed -- trouble finding a new table after dropping old
...
create_index .. failed -- trouble creating an index after creating first
sanity_check .. failed -- missing tables from previous failures, n/a
...
select_views .. failed -- old (one month?) problem with core dump
...
run_ruletest .. failed -- dba account name differs, n/a
I did a "make distclean" earlier this evening, and a "make clean" and
"initdb" during this recent build session, so things should be up to
date. Let's not worry about select_views until we've fixed constraints
and create_index, since that is an older problem.
I'm sure we are getting closer though...
- Tom
Thomas G. Lockhart wrote:
I am still waiting for the first report from my index fix.
Anyone?OK, here is my interim report: everything is broken :)
Looks like David's patches busted several Makefiles. Will poke at it and
let you know.
Did this get resolved? I reviewed the original patch, and the missing
items that are mentioned in this thread are in the patch.
Did this get resolved?
Yes, a file got left out of the cvs commit; Bruce fixed it right away.
Update:
I'm still seeing problems with the regression tests, and it appears to
be the same symptom reported by someone else earlier: a couple of tables
(or indices) exist but something is munged in pg_class so that I can
only see the entry using a "like" query; an "=" equals query does not
return the row.
This may only happen with tables which have been destroyed and then
redefined??
I've done a full "cvs update -Pd", a "make distclean", and an initdb.
I'll try a clean checkout next. If that doesn't help, then will start
poking at it...
- Tom
Thomas G. Lockhart wrote:
Did this get resolved?
Yes, a file got left out of the cvs commit; Bruce fixed it right away.
Update:
I'm still seeing problems with the regression tests, and it appears to
be the same symptom reported by someone else earlier: a couple of tables
(or indices) exist but something is munged in pg_class so that I can
only see the entry using a "like" query; an "=" equals query does not
return the row.
I know why this was happening. (At least on the surface) In my case,
whenever a I added an index to a table pg_class_relname_index was getting
corrupted. The nature of the corruption was that any query that used the
pg_class_relname_index to find a table that was just indexed, could no
longer find it. The corruption must occur on the update of pg_class when
the index is added. This explains why:
CREATE TABLE foo (i int);
CREATE INDEX foo_idx ON foo USING btree(i);
SELECT * FROM pg_class;
Showed a complete correct list of tables.
SELECT * FROM pg_class WHERE = 'foo';
and
SELECT * FROM pg_class WHERE LIKE 'foo%';
Showed nothing. I did not know LIKE was using an index. It through
me. Nice job by sombody.
SELECT * FROM pg_class WHERE LIKE '%foo';
Showed my original table. But this query like the first does a full
scan.
In any case I will be doing some testing myself this afternoon.
OK, everything is there now.
I did a CVSup, a cvs checkout into a new tree, and still see the same
problems. I'll bet that most of the symptoms in the regression tests are
due to the same thing, which seems to involve table munging of pg_class.
It looks like some tuples end up with leading garbage of some sort in
the first field: see the example below.
I'll try to distill this down to a simpler test case. If anyone has some
hints on where to look...
- Tom
regression=> select * from primary_tbl;
ERROR: primary_tbl: Table does not exist.
regression=> select * from pg_class where relname = 'primary_tbl';
relname|reltype|relowner|relam|relpages|reltuples|relhasindex|relisshared|relkind|relnatts|relchecks|reltriggers|relhasrules|relacl
-------+-------+--------+-----+--------+---------+-----------+-----------+-------+--------+---------+-----------+-----------+------
(0 rows)
regression=> select * from pg_class where relname like '%primary_tbl%';
relname
|reltype|relowner|relam|relpages|reltuples|relhasindex|relisshared|relkind|relnatts|relchecks|reltriggers|relhasrules|relacl
----------------+-------+--------+-----+--------+---------+-----------+-----------+-------+--------+---------+-----------+-----------+------
primary_tbl | 0| 500| 0| 0| 0|t
|f |r | 2| 0| 0|f |
primary_tbl_pkey| 0| 500| 403| 2| 1024|f
|f |i | 2| 0| 0|f |
(2 rows)
regression=> select * from pg_class where relname like 'primary_tbl%';
relname|reltype|relowner|relam|relpages|reltuples|relhasindex|relisshared|relkind|relnatts|relchecks|reltriggers|relhasrules|relacl
-------+-------+--------+-----+--------+---------+-----------+-----------+-------+--------+---------+-----------+-----------+------
(0 rows)
regression=> select * from pg_class where relname like '%primary_tbl';
relname
|reltype|relowner|relam|relpages|reltuples|relhasindex|relisshared|relkind|relnatts|relchecks|reltriggers|relhasrules|relacl
-----------+-------+--------+-----+--------+---------+-----------+-----------+-------+--------+---------+-----------+-----------+------
primary_tbl| 0| 500| 0| 0| 0|t
|f |r | 2| 0| 0|f |
(1 row)
Thomas G. Lockhart wrote:
I am still waiting for the first report from my index fix.
Anyone?OK, here is my interim report: everything is broken :)
Looks like David's patches busted several Makefiles. Will poke at it and
let you know.Did this get resolved? I reviewed the original patch, and the missing
items that are mentioned in this thread are in the patch.
Resolved. I forgot to add the new file.
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)Tom, its in the index. Do an explain before each of your queries.
Thomas G. Lockhart wrote:
Show quoted text
OK, everything is there now.
I did a CVSup, a cvs checkout into a new tree, and still see the same
problems. I'll bet that most of the symptoms in the regression tests are
due to the same thing, which seems to involve table munging of pg_class.
It looks like some tuples end up with leading garbage of some sort in
the first field: see the example below.I'll try to distill this down to a simpler test case. If anyone has some
hints on where to look...- Tom
regression=> select * from primary_tbl;
ERROR: primary_tbl: Table does not exist.regression=> select * from pg_class where relname = 'primary_tbl';
relname|reltype|relowner|relam|relpages|reltuples|relhasindex|relisshared|relkind|relnatts|relchecks|reltriggers|relhasrules|relacl
-------+-------+--------+-----+--------+---------+-----------+-----------+-------+--------+---------+-----------+-----------+------
(0 rows)regression=> select * from pg_class where relname like '%primary_tbl%';
relname
|reltype|relowner|relam|relpages|reltuples|relhasindex|relisshared|relkind|relnatts|relchecks|reltriggers|relhasrules|relacl
----------------+-------+--------+-----+--------+---------+-----------+-----------+-------+--------+---------+-----------+-----------+------
primary_tbl | 0| 500| 0| 0| 0|t
|f |r | 2| 0| 0|f |
primary_tbl_pkey| 0| 500| 403| 2| 1024|f
|f |i | 2| 0| 0|f |
(2 rows)regression=> select * from pg_class where relname like 'primary_tbl%';
relname|reltype|relowner|relam|relpages|reltuples|relhasindex|relisshared|relkind|relnatts|relchecks|reltriggers|relhasrules|relacl
-------+-------+--------+-----+--------+---------+-----------+-----------+-------+--------+---------+-----------+-----------+------
(0 rows)regression=> select * from pg_class where relname like '%primary_tbl';
relname
|reltype|relowner|relam|relpages|reltuples|relhasindex|relisshared|relkind|relnatts|relchecks|reltriggers|relhasrules|relacl
-----------+-------+--------+-----+--------+---------+-----------+-----------+-------+--------+---------+-----------+-----------+------
primary_tbl| 0| 500| 0| 0| 0|t
|f |r | 2| 0| 0|f |
(1 row)
Thomas G. Lockhart wrote:
OK, everything is there now.
I did a CVSup, a cvs checkout into a new tree, and still see the same
problems.
Forgot to mention. I still have the problem also. Tom, what are you running on? Is platform still a factor in this problem?
In message <35EEA3A6.8E9270F7@alumni.caltech.edu>, "Thomas G. Lockhart" writes:
Did this get resolved?
Yes, a file got left out of the cvs commit; Bruce fixed it right away.
Update:
I'm still seeing problems with the regression tests, and it appears to
be the same symptom reported by someone else earlier: a couple of tables
(or indices) exist but something is munged in pg_class so that I can
only see the entry using a "like" query; an "=" equals query does not
return the row.This may only happen with tables which have been destroyed and then
redefined??I've done a full "cvs update -Pd", a "make distclean", and an initdb.
I'll try a clean checkout next. If that doesn't help, then will start
poking at it...- Tom
I just did a cvs update, and ran the regression test on solaris.
I think what Tom is describes is my only remaining problem.
Tables just appear to "vanish". For instance this is from the
triggers test:
QUERY: create table pkeys (pkey1 int4 not null, pkey2 text not null);
QUERY: create table fkeys (fkey1 int4, fkey2 text, fkey3 int);
QUERY: create table fkeys2 (fkey21 int4, fkey22 text, pkey23 int not null);
QUERY: create index fkeys_i on fkeys (fkey1, fkey2);
QUERY: create index fkeys2_i on fkeys2 (fkey21, fkey22);
QUERY: create index fkeys2p_i on fkeys2 (pkey23);
ERROR: DefineIndex: fkeys2 relation not found
Odd that it works the first time. The triggers test later crashes.
I also had this at to top of sanity_check:
QUERY: VACUUM;
NOTICE: Index pg_class_relname_index: NUMBER OF INDEX' TUPLES (169) IS NOT THE SAME AS HEAP' (139)
NOTICE: Index pg_class_oid_index: NUMBER OF INDEX' TUPLES (169) IS NOT THE SAME AS HEAP' (139)
NOTICE: Rel pg_trigger: Uninitialized page 0 - fixing
All in all *much* better!
Thanks,
Tom Szybist
szybist@boxhill.com
Forgot to mention. I still have the problem also. Tom, what are
you running on? Is platform still a factor in this problem?
Platform is a problem in that Bruce's FreeBSD/i686 machine does not show
the symptoms, but it's a pretty widespread problem beyond that afaik.
I'm running on Linux/i686. Tatsuo sees problems on Linux/PPC. Some Sparc
machines running Solaris and Linux apparently still show problems.
I'm guessing that it is a byte alignment difference in malloc behavior
between the systems which exposes misaligned structures on some
platforms. How's that for pure speculation, eh?
- Tom
Forgot to mention. I still have the problem also. Tom, what are
you running on? Is platform still a factor in this problem?Platform is a problem in that Bruce's FreeBSD/i686 machine does not show
the symptoms, but it's a pretty widespread problem beyond that afaik.
I'm running on Linux/i686. Tatsuo sees problems on Linux/PPC. Some Sparc
machines running Solaris and Linux apparently still show problems.I'm guessing that it is a byte alignment difference in malloc behavior
between the systems which exposes misaligned structures on some
platforms. How's that for pure speculation, eh?
Let me tell you what I need to help debug this.
It would help to know if it is the cache, or an index problem. It is
sometimes hard to determine because the cache often uses the indexes to
load the cache.
Can someone step through a bad entry, and tell me where it is failing?
If it is in the executor, it probably is an index. EXPLAIN does show
what indexes are involved. Are several indexes failing, or just one?
I like the malloc idea, but am unsure how the problem just started
happening with the multi-key system indexes.
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)
Bruce Momjian wrote:
Forgot to mention. I still have the problem also. Tom, what are
you running on? Is platform still a factor in this problem?Platform is a problem in that Bruce's FreeBSD/i686 machine does not show
the symptoms, but it's a pretty widespread problem beyond that afaik.
I'm running on Linux/i686. Tatsuo sees problems on Linux/PPC. Some Sparc
^^^^^^^^^^
machines running Solaris and Linux apparently still show problems.
^^^^^^^^^^^^^^^^^^^^^^^^
This is what I have:
SunOS sunpine.krs.ru 5.5.1 Generic_103640-12 sun4u sparc SUNW,Ultra-4
I didn't install 6.4 on this machine but could to do this...
What are the problems ?
Vadim
