Lossy bitmap scan is broken in GIN
Hackers,
GIN hangs on lossy bitmap scan. Here is test case:
create extension btree_gin;
create table test as (select random() v from generate_series(1,1000000));
create index test_idx on test using gin(v);
set work_mem = '16MB';
select count(*) from test where v > 0.9;
count
───────
99916
(1 row)
Time: 63,142 ms
set work_mem = '64kB';
select count(*) from test where v > 0.9;
The last query hangs. I've debugged it. It's another bug in this "cursed"
loop in entryGetItem. Fix is attached.
------
With best regards,
Alexander Korotkov.
Attachments:
gin_lossy_bitmap_fix.patchapplication/octet-stream; name=gin_lossy_bitmap_fix.patchDownload
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
new file mode 100644
index 271f099..144ed50
*** a/src/backend/access/gin/ginget.c
--- b/src/backend/access/gin/ginget.c
*************** entryGetItem(GinState *ginstate, GinScan
*** 741,747 ****
while (entry->matchResult == NULL ||
(entry->matchResult->ntuples >= 0 &&
entry->offset >= entry->matchResult->ntuples) ||
! entry->matchResult->blockno < advancePastBlk)
{
entry->matchResult = tbm_iterate(entry->matchIterator);
--- 741,749 ----
while (entry->matchResult == NULL ||
(entry->matchResult->ntuples >= 0 &&
entry->offset >= entry->matchResult->ntuples) ||
! entry->matchResult->blockno < advancePastBlk ||
! (ItemPointerIsLossyPage(&advancePast) &&
! entry->matchResult->blockno == advancePastBlk))
{
entry->matchResult = tbm_iterate(entry->matchIterator);
On 05/10/2014 10:41 PM, Alexander Korotkov wrote:
Hackers,
GIN hangs on lossy bitmap scan. Here is test case:
create extension btree_gin;
create table test as (select random() v from generate_series(1,1000000));
create index test_idx on test using gin(v);
set work_mem = '16MB';
select count(*) from test where v > 0.9;
count
───────
99916
(1 row)Time: 63,142 ms
set work_mem = '64kB';
select count(*) from test where v > 0.9;The last query hangs. I've debugged it. It's another bug in this "cursed"
loop in entryGetItem. Fix is attached.
Hmm, so if we're scanning a bitmap, as in a partial match query, and the
bitmap contains a lossy page reference, and advancePast is set to a
lossy reference to the same page (as it usually will be), we get stuck
in an infinite loop. The first while-loop is responsible for finding the
right block, and doesn't look at the offsets, so it stops on that block.
But the if-test that checks if it's a lossy page doesn't check if
advancePast was also a lossy reference, so it returns it again.
Committed, thanks!
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers