BUG #19582: Query fails on mixed IPv4 IPv6 data when index added
The following bug has been logged on the website:
Bug reference: 19582
Logged by: Giorgio Saviane
Email address: gsaviane@gmail.com
PostgreSQL version: 17.10
Operating system: Ubuntu 22.04
Description:
The attached SQL code snippet shows how to reproduce the problem. A table
with generated mixed IPv4 and IPv6 data, when queried filtering by IP family
fails when a conditional index is being added to the table. I suspect the
planner pre-executes the query on some samples, applying the index condition
only partially.
This is the output I get when executing the SQL:
BEGIN
CREATE TABLE
INSERT 0 20000000
count
---------
8355840
(1 row)
CREATE INDEX
QUERY PLAN
-----------------------------------------------------------------------------------------------
Aggregate (cost=1913.76..1913.77 rows=1 width=8)
-> Bitmap Heap Scan on test_inet (cost=8.31..1912.51 rows=500 width=0)
Recheck Cond: (((addr & '255.0.0.0'::inet) = '1.0.0.0'::inet) AND
(family(addr) = 4))
-> Bitmap Index Scan on test_inet_expr_idx (cost=0.00..8.18
rows=500 width=0)
Index Cond: ((addr & '255.0.0.0'::inet) = '1.0.0.0'::inet)
(5 rows)
ERROR: cannot AND inet values of different sizes
ROLLBACK
=============== CODE HERE ==============
begin;
-- Create a table to host inet addresses
create table test_inet (
addr inet not null
);
-- Generate mixed IPv4 and IPv6 data.
-- The amout of data makes the difference.
-- Experimentally, lower than 13M it does not reproduce.
-- It might depend on server configuration
insert into test_inet(addr)
select (case
mod(s,2) when 0
then 'fe80:1::'::inet + s
else '1.1.0.0'::inet + s
end) from generate_series(1, 20000000) s;
-- Count filtered by IPv4 family and subnet.
-- Suceeds without an index
select count(*)
from test_inet
where family(addr) = 4
and addr & '255.0.0.0'::inet = '1.0.0.0';
-- Now create an index filtered only on IPv4 family
create index
on test_inet((addr & '255.0.0.0'::inet))
where family(addr) = 4;
-- The plan should show how subnet and family
-- conditions are split due to rechecking
explain select count(*)
from test_inet
where family(addr) = 4
and addr & '255.0.0.0'::inet = '1.0.0.0';
-- Same count filtered by IPv4 family and subnet fails with:
-- ERROR: cannot AND inet values of different sizes
select count(*)
from test_inet
where family(addr) = 4
and addr & '255.0.0.0'::inet = '1.0.0.0';
rollback;
=============== END CODE ==============
PG Bug reporting form <noreply@postgresql.org> writes:
The attached SQL code snippet shows how to reproduce the problem. A table
with generated mixed IPv4 and IPv6 data, when queried filtering by IP family
fails when a conditional index is being added to the table. I suspect the
planner pre-executes the query on some samples, applying the index condition
only partially.
I'm not seeing any particular bug here. The & operator doesn't work
on mixed address widths:
regression=# select '::1'::inet & '127.0.0.1'::inet;
ERROR: cannot AND inet values of different sizes
so applying it to a data column that contains mixed widths is
inherently dangerous.
You appear to be hoping that going via the partial index will prevent
applying the operator to IPv6 addresses, but that is not bulletproof
and was never claimed to be. In particular, once there's enough data
the planner will probably use a bitmap index scan, which is lossy ---
at some point it's going to report something like "these page(s)
contain matching tuples", leaving it to the main executor to scan
all the tuples on those pages. If any of those are IPv6, kaboom.
A safer answer might be to partition the table between IPv4 and IPv6
addresses. If the planner can match the query condition to the
partitioning rule, it won't scan the non-matching partition at all.
I wonder if it'd make more sense to have & promote the IPv4 address
to IPv6 and then perform ANDing, rather than failing outright.
I think when these operators were extended to IPv6, it wasn't entirely
clear what the appropriate widening rule was, but surely that's been
resolved by now.
regards, tom lane