BUG #19638: Planner chooses an index-only scan for an index AM without amcanreturn, and execution fails
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t253529psql -h localhost -U postgresBuilt from patchset v7 (message #7), September 15, 2026 at 03:24 PM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t253529_7 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t253529_7 && git checkout t253529_7Patchset v7 (message #7) is on t253529_7
The following bug has been logged on the website:
Bug reference: 19638
Logged by: Manuel Reyes Bravo
Email address: manuelreyesbravo@gmail.com
PostgreSQL version: 19beta3
Operating system: Fedora 44, Linux 7.1.8, gcc 16.1.1, PostgreSQL bui
Description:
Note up front: reproducing this needs a third-party index access method, but
the bug itself is in core, not in the extension. An AM that does not
implement
amcanreturn is legal per the documented index AM API; the planner
nevertheless
builds an index-only scan over it, and the executor then cannot run the
plan.
The extension is only the vehicle that exposes it -- I could not find any
in-core AM with the required combination (see "Why no in-core reproducer"
below), which is probably why this has gone unnoticed.
On PostgreSQL 19beta3 the following query produces a plan that cannot be
executed:
ERROR: no data returned for index-only scan
The same query, same schema and same extension code works correctly on 18.6.
Reproducer
----------
Using pgvectorscale 0.9.0 (its "diskann" AM) with pgvector 0.8.6:
CREATE EXTENSION vector;
CREATE EXTENSION vectorscale;
CREATE TABLE t_nopk (embedding vector(3));
CREATE INDEX idx_nopk ON t_nopk USING diskann (embedding);
INSERT INTO t_nopk VALUES ('[1,2,3]'), ('[4,5,6]'), ('[7,8,9]');
SET enable_seqscan = 0;
SELECT COUNT(*)
FROM (SELECT embedding FROM t_nopk ORDER BY embedding <-> NULL LIMIT
3) x;
19beta3:
QUERY PLAN
---------------------------------------------------
Aggregate
-> Limit
-> Index Only Scan using idx_nopk on t_nopk
ERROR: no data returned for index-only scan
18.6 (same extension, same schema, same query):
QUERY PLAN
---------------------------------
Aggregate
-> Limit
-> Seq Scan on t_nopk
Disabled: true
count
-------
3
So 18 correctly falls back to a disabled sequential scan and returns the
right
answer, while 19 produces an unexecutable plan.
Note: the table must have no PRIMARY KEY
----------------------------------------
With a btree primary key present, the planner uses that index for the
index-only scan instead and the problem does not appear. That cost me some
time, so it may save yours.
Why there is no in-core reproducer
----------------------------------
I tried to reproduce this with in-core AMs and could not. GIN and hash also
lack amcanreturn, but they require an index qual, so the path is never
considered. It appears to need amoptionalkey = true together with a missing
amcanreturn, and as far as I can tell no in-core AM has that combination. A
regression test would probably have to go through a test module.
Where it seems to come from
---------------------------
check_index_only() in src/backend/optimizer/path/indxpath.c ends with
return bms_is_subset(attrs_used, index_canreturn_attrs);
When attrs_used is empty, bms_is_subset() returns true regardless of what
the
AM can actually return, while index_can_return() returns false for an AM
whose
amcanreturn is NULL. So an index that can return nothing at all passes the
check as long as the query needs no attributes from it.
I have not bisected this, so what follows is a guess rather than a finding:
indxpath.c gained a path-generation mask in "Allow for plugin control over
path
generation strategies" (2026-01-28), and PGS_CONSIDER_INDEXONLY looks like a
plausible reason why this path is now considered where it previously was
not.
Someone familiar with that code will see it much faster than I did.
This also looks related to the earlier discussion in "[PATCH] Check that
index
can return in get_actual_variable_range()" (Sept-Oct 2025), which addressed
the
same underlying assumption in a different place. This case does not appear
to
be covered by that fix.
Versions tested
---------------
PostgreSQL 19beta3, built from source: fails as shown above
PostgreSQL 18.6, built from source with the same compiler and flags:
correct
Happy to test a patch or provide any further detail.
On Tue, 25 Aug 2026 at 00:16, PG Bug reporting form
<noreply@postgresql.org> wrote:
Note up front: reproducing this needs a third-party index access method, but
the bug itself is in core, not in the extension. An AM that does not
implement
amcanreturn is legal per the documented index AM API; the planner
nevertheless
builds an index-only scan over it, and the executor then cannot run the
plan.
The extension is only the vehicle that exposes it -- I could not find any
in-core AM with the required combination (see "Why no in-core reproducer"
below), which is probably why this has gone unnoticed.
What do you mean by "required combination"? We have plenty of
IndexAMs that don't implement amcanreturn, e.g. brin.c.
Going by what you've reported in bug #19639, I'm suspecting you might
have done something to mix up the binaries for the extension.
I tried to test this, but pgvectorscale doesn't seem to support pg19:
drowley@amd3990x:~/pgvectorscale/pgvectorscale$ cargo pgrx install --release
Using PgConfig("pg19") and `pg_config` from
/home/drowley/pg/bin/pg_config
Building extension with features build_parallel pg19
Running command
"/home/drowley/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo"
"build" "--lib" "--release" "--features" "build_parallel pg19"
"--no-default-features" "--message-format=json-render-diagnostics"
error: the package 'vectorscale' does not contain this feature: pg19
help: there are similarly named features: pg14, pg15, pg16, pg17, pg18
Did you compile it yourself? How?
David
пн, 24 авг. 2026 г. в 19:05, David Rowley <dgrowleyml@gmail.com>:
What do you mean by "required combination"? We have plenty of
IndexAMs that don't implement amcanreturn, e.g. brin.c.
Dear David,
"No amcanreturn" alone is not enough.
The failure needs
amoptionalkey = true together with amgettuple != NULL and amcanreturn
NULL. I could not find an in-core AM with that combination; diskann
has it, which is why the third-party extension showed the bug.
https://github.com/timescale/pgvectorscale/blob/main/pgvectorscale/src/access_method/mod.rs
```
amroutine.amoptionalkey = true;
...
amroutine.amgettuple = Some(scan::amgettuple);
amroutine.amgetbitmap = None;
```
I reproduced it on current master without pgvectorscale, using a tiny
module AM that only implements those flags and returns heap TIDs
without filling xs_hitup/xs_itup. With enable_seqscan/bitmapscan off:
```
Aggregate
-> Index Only Scan using idx_nopk on t_nopk
ERROR: no data returned for index-only scan
```
So this is not a mixed-binary problem with the extension.
The hole is in check_index_only(): bms_is_subset(attrs_used,
index_canreturn_attrs) is true when attrs_used is empty even if the
index cannot return any column. That matches the empty-targetlist
count(*) case. It is in the same family as the earlier
get_actual_variable_range() amcanreturn check (74197bdc842).
--
Regards,
Rachitskiy Andrey
Andrey -- thank you for the independent confirmation, and for the pointer
to 74197bdc842, which is a better reference than the thread title I had.
David: re-attaching nokeyam.tar.gz here in case it did not survive my
previous message. Same module as described: dummy_index_am plus
amoptionalkey = true, an amgettuple returning one heap TID, and
amcanreturn left NULL.
One thing I have not seen mentioned yet, which may matter for the fix and
for whatever test you settle on: the trigger is not count(*), it is any
query whose target list needs no attribute from the index. Measured on
18.6 with the module above, seqscan and bitmapscan off:
SELECT COUNT(*) FROM t_nokey; Index Only Scan -> ERROR
SELECT FROM t_nokey; Index Only Scan -> ERROR
SELECT EXISTS (SELECT 1 FROM t_nokey); Index Only Scan -> ERROR
SELECT a FROM t_nokey; Seq Scan (disabled) -> correct
SELECT b FROM t_nokey; Seq Scan (disabled) -> correct
where a is the indexed column and b is not. So as soon as anything is
actually requested from the relation, attrs_used is non-empty, the subset
test does its job and the path is rejected -- which lines up with the
empty-set reading of check_index_only() that the three of us arrived at
separately.
EXISTS seems worth noting because it shows up in ordinary code through
semi-joins, without anyone writing count(*).
The attached tarball includes that script as alcance.sql.
Happy to test a patch on both branches.
Attachments:
пн, 24 авг. 2026 г. в 21:51, Manuel Reyes Bravo <manuelreyesbravo@gmail.com
:
The attached tarball includes that script as alcance.sql.
In the future, it would be better to attach patches rather than archives.
I kept the fix minimal: one guard after bms_is_subset() in
check_index_only(), rejecting the plan when no key column is returnable.
--
Regards,
Rachitskiy Andrey
Thanks for the patch. I tested it on REL_18_STABLE-equivalent sources (18.6,
built from source), since that is the branch this would need to be
back-patched to. Short version: it fixes the bug, it passes the full
regression suite, and I believe it also rejects a legitimate plan.
What works
----------
Applied cleanly (hunk offset 2 lines). With the test module:
before: Index Only Scan using i_nokey -> ERROR: no data returned...
after: Seq Scan (disabled) -> 3 correct
make check: all 231 tests passed.
What I think is a false positive
--------------------------------
The guard keys off bms_is_empty(index_canreturn_attrs), but that bitmapset
is
also empty for an index whose columns are all expressions, because the
loop
just above skips them:
/*
* For the moment, we just ignore index expressions. It might be nice
* to do something with them, later.
*/
if (attno == 0)
continue;
So "empty" does not mean "the AM can return nothing", it means "no plain
columns are returnable". A btree over an expression can feed an index-only
scan perfectly well. Measured on 18.6, with enable_seqscan off:
CREATE TABLE t_expr (a int, b int);
INSERT INTO t_expr SELECT g, g*2 FROM generate_series(1,50000) g;
CREATE INDEX i_expr ON t_expr ((a + b));
VACUUM ANALYZE t_expr;
SELECT count(*) FROM t_expr;
unpatched: Aggregate -> Index Only Scan using i_expr on t_expr
patched: Aggregate -> Seq Scan on t_expr (disabled)
Both return 50000, so this is a plan regression rather than a correctness
one -- counting can no longer walk the smaller index. A control with an
ordinary column index (CREATE INDEX i_col ON t_col (a)) keeps its index-only
scan under the patch, so the effect is specific to expression-only indexes.
Worth noting: make check does not catch this. The suite passed 231/231 with
the patch applied, so this would go in unnoticed.
A variant that avoids it
------------------------
Attached as a patch this time, rather than an archive. It tests the AM's
capability directly instead of the bitmapset:
if (result)
{
bool any_canreturn = false;
for (i = 0; i < index->ncolumns; i++)
{
if (index->canreturn[i])
{
any_canreturn = true;
break;
}
}
if (!any_canreturn)
result = false;
}
index->canreturn[] is filled per column from index_can_return() in
plancat.c,
including expression columns, so an expression btree has a true entry
while an
AM with amcanreturn == NULL has none.
Measured on 18.6 with that variant:
the reproducer -> 3, correct (bug fixed)
count(*) over i_expr -> Index Only Scan (no regression)
count(*) over i_col -> Index Only Scan (unchanged)
make check -> all 231 tests passed
The patch is against 18.6 sources, since that is what I tested on; it should
apply to master with an offset.
I have not tried to judge which shape you would prefer, and there may be a
reason to keep it keyed off the bitmapset that I am not seeing. I can rerun
any of this on 19beta2 as well if that is useful.
El lun, 24 ago 2026 a las 14:20, Andrey Rachitskiy (<pl0h0yp1@gmail.com>)
escribió:
пн, 24 авг. 2026 г. в 21:51, Manuel Reyes Bravo <
manuelreyesbravo@gmail.com>:The attached tarball includes that script as alcance.sql.
In the future, it would be better to attach patches rather than archives.
I kept the fix minimal: one guard after bms_is_subset() in
check_index_only(), rejecting the plan when no key column is returnable.--
Regards,
Rachitskiy Andrey
--
Saludos cordiales,
Manuel Reyes
Attachments:
v2-0001-Reject-index-only-scans-when-the-AM-can-return-nothing.patchtext/x-patch; charset=US-ASCII; name=v2-0001-Reject-index-only-scans-when-the-AM-can-return-nothing.patchDownload+25-0
пн, 24 авг. 2026 г. в 23:52, Manuel Reyes Bravo <manuelreyesbravo@gmail.com
:
unpatched: Aggregate -> Index Only Scan using i_expr on t_expr
patched: Aggregate -> Seq Scan on t_expr (disabled)
Agreed, v3 fixed this.
--
Regards,
Rachitskiy Andrey
Import Notes
Reply to msg id not found: CA+bCEdD77EUEsm1+WDQM=m9rcgvibOKv9A9uz+Fg46Yj+zk=Kw@mail.gmail.com
Agreed, v3 fixed this.
Confirmed on 18.6. v3 applies with a 2-line offset and passes everything I
threw at the previous one:
the #19638 reproducer Seq Scan (disabled) -> 3, correct
count(*) over i_expr Index Only Scan (kept)
count(*) over i_col Index Only Scan (unchanged)
SELECT FROM t_nokey Seq Scan (disabled), correct
EXISTS (SELECT 1 FROM ...) Seq Scan (disabled), correct
make check all 231 tests passed
Tracking any_canreturn inside the existing loop is nicer than the separate
pass I suggested -- one traversal, and it reads as part of building the
bitmap rather than as an afterthought.
Nothing further from me on this one.
El lun, 24 ago 2026 a las 15:34, Andrey Rachitskiy (<pl0h0yp1@gmail.com>)
escribió:
пн, 24 авг. 2026 г. в 23:52, Manuel Reyes Bravo <
manuelreyesbravo@gmail.com>:
unpatched: Aggregate -> Index Only Scan using i_expr on t_expr
patched: Aggregate -> Seq Scan on t_expr (disabled)Agreed, v3 fixed this.
--
Regards,
Rachitskiy Andrey
--
Saludos cordiales,
Manuel Reyes