BUG #19829: BTREE pivot tuple length overflows t_info when BLCKSZ=32768
The following bug has been logged on the website:
Bug reference: 19829
Logged by: wangfei
Email address: wf13934952799@gmail.com
PostgreSQL version: 18.6
Operating system: ubuntu-24.04
Description:
PostgreSQL version: 19devel (also reproducible on 18.x)
Operating system: Linux x86_64
Configure options: --with-blocksize=32
With BLCKSZ=32768, BTMaxItemSize() evaluates to roughly 10896 bytes, but the
tuple length field reserved by INDEX_SIZE_MASK in IndexTupleData.t_info can
only represent 8191 bytes. index_form_tuple() enforces the 8191 limit, so a
leaf tuple is at most 8184 bytes after MAXALIGN. _bt_truncate() may then
need
to append a tiebreaker heap TID:
newsize = MAXALIGN(IndexTupleSize(pivot)) +
MAXALIGN(sizeof(ItemPointerData));
...
tidpivot->t_info &= ~INDEX_SIZE_MASK;
tidpivot->t_info |= newsize;
For a maximally sized leaf tuple this is 8184 + 8 = 8192, i.e. exactly
INDEX_SIZE_MASK + 1, so the length field wraps to zero. The result is a
zero-length pivot tuple used as a leaf high key; PageAddItem() and
XLogRegisterBufData() both accept it without complaint, so the corruption is
written out and WAL-logged silently.
BTMaxItemSize() is supposed to keep leaf tuples small enough that this
cannot
happen, but the one-third-of-a-page rule scales with BLCKSZ while
INDEX_SIZE_MASK does not, so at BLCKSZ=32768 the one-third rule is no longer
the binding constraint. Block sizes of 16384 and below are unaffected.
Steps to reproduce:
< SQL 脚本>
Expected: either the insert is rejected, or the pivot tuple is stored
correctly.
Actual: high key with length 0 (amcheck reports: <具体报错>).
A patch against master is attached. It takes Min() of the one-third rule
and
what t_info can represent in BTMaxItemSize(), and adds a defensive
elog(ERROR)
in _bt_truncate() so that a future miscalculation cannot corrupt an index
silently. Since index_form_tuple() already caps index tuples at 8184 bytes,
lowering the BLCKSZ=32768 limit cannot invalidate any index that could
actually exist on disk, so there is no on-disk compatibility concern.
CREATE EXTENSION amcheck;
-- One incompressible 8172-byte value. The resulting index tuple is
-- MAXALIGN(sizeof(IndexTupleData) + VARHDRSZ + 8172) =
MAXALIGN(8+4+8172) = 8184
-- bytes, which is the largest index tuple index_form_tuple() will ever
produce.
CREATE TABLE pad AS
SELECT substr((SELECT string_agg(md5(random()::text), '')
FROM generate_series(1, 300)), 1, 8172) AS v;
CREATE TABLE t (v text);
-- Must come before CREATE INDEX: the index inherits attstorage, and this
stops
-- TOAST_INDEX_HACK in index_form_tuple() from compressing the value.
ALTER TABLE t ALTER COLUMN v SET STORAGE EXTERNAL;
CREATE INDEX t_v_idx ON t (v);
-- Four identical keys. Three fill the leaf page; the fourth splits it.
Since
-- all keys are equal, _bt_truncate() cannot truncate away any key
attribute and
-- must append a tiebreaker heap TID: newsize = 8184 + 8 = 8192.
INSERT INTO t SELECT v FROM pad, generate_series(1, 4);
SELECT bt_index_check('t_v_idx', true);
ERROR: invalid line pointer storage in index "t_v_idx"
DETAIL: Index tid=(3,2) lp_off=32744, lp_len=0 lp_flags=1.
PG Bug reporting form <noreply@postgresql.org> 于2026年11月3日周二 15:47写道:
Show quoted text
The following bug has been logged on the website:
Bug reference: 19829
Logged by: wangfei
Email address: wf13934952799@gmail.com
PostgreSQL version: 18.6
Operating system: ubuntu-24.04
Description:PostgreSQL version: 19devel (also reproducible on 18.x)
Operating system: Linux x86_64
Configure options: --with-blocksize=32With BLCKSZ=32768, BTMaxItemSize() evaluates to roughly 10896 bytes, but
the
tuple length field reserved by INDEX_SIZE_MASK in IndexTupleData.t_info can
only represent 8191 bytes. index_form_tuple() enforces the 8191 limit, so
a
leaf tuple is at most 8184 bytes after MAXALIGN. _bt_truncate() may then
need
to append a tiebreaker heap TID:newsize = MAXALIGN(IndexTupleSize(pivot)) +
MAXALIGN(sizeof(ItemPointerData));
...
tidpivot->t_info &= ~INDEX_SIZE_MASK;
tidpivot->t_info |= newsize;For a maximally sized leaf tuple this is 8184 + 8 = 8192, i.e. exactly
INDEX_SIZE_MASK + 1, so the length field wraps to zero. The result is a
zero-length pivot tuple used as a leaf high key; PageAddItem() and
XLogRegisterBufData() both accept it without complaint, so the corruption
is
written out and WAL-logged silently.BTMaxItemSize() is supposed to keep leaf tuples small enough that this
cannot
happen, but the one-third-of-a-page rule scales with BLCKSZ while
INDEX_SIZE_MASK does not, so at BLCKSZ=32768 the one-third rule is no
longer
the binding constraint. Block sizes of 16384 and below are unaffected.Steps to reproduce:
< SQL 脚本>Expected: either the insert is rejected, or the pivot tuple is stored
correctly.
Actual: high key with length 0 (amcheck reports: <具体报错>).A patch against master is attached. It takes Min() of the one-third rule
and
what t_info can represent in BTMaxItemSize(), and adds a defensive
elog(ERROR)
in _bt_truncate() so that a future miscalculation cannot corrupt an index
silently. Since index_form_tuple() already caps index tuples at 8184
bytes,
lowering the BLCKSZ=32768 limit cannot invalidate any index that could
actually exist on disk, so there is no on-disk compatibility concern.
#!/bin/bash
#
=============================================================================
# B-tree pivot tuple length overflows t_info when BLCKSZ = 32768
#
# BTMaxItemSize() scales with BLCKSZ (10896 at 32kB pages), but the tuple
# length field reserved by INDEX_SIZE_MASK in IndexTupleData.t_info is a
fixed
# 13 bits (8191). index_form_tuple() caps index tuples at 8184 bytes after
# MAXALIGN, so when _bt_truncate() has to append a tiebreaker heap TID it
# computes newsize = 8184 + 8 = 8192 = INDEX_SIZE_MASK + 1 and
# `pivot->t_info |= newsize` wraps the length field to zero.
#
# Needs a build with --with-blocksize=32. At BLCKSZ 8192/16384 the
# one-third-of-a-page rule (2704/5432) binds first, so those are
unaffected.
#
=============================================================================
set -u
PGBIN=${PGBIN:-/home/coder/postgres_github/release/REL_18_STABLE_32/bin}
PGDATA=${PGDATA:-/tmp/pgdata32_repro}
SOCKDIR=${SOCKDIR:-/tmp/s32repro} # keep short: socket path limit is 107
bytes
PGPORT=${PGPORT:-55432}
PSQL="$PGBIN/psql -h $SOCKDIR -p $PGPORT -U postgres -d postgres"
echo "### 0. confirm the binary uses 32kB pages"
$PGBIN/pg_config --configure | tr ' ' '\n' | grep blocksize || {
echo "!! not a 32kB build, cannot reproduce"; exit 1; }
echo "### 1. init + start"
rm -rf "$PGDATA"; mkdir -p "$SOCKDIR"
$PGBIN/initdb -D "$PGDATA" --locale=C --encoding=UTF8 -U postgres >/dev/null
2>&1
cat >> "$PGDATA/postgresql.conf" <<EOF
port = $PGPORT
listen_addresses = ''
unix_socket_directories = '$SOCKDIR'
fsync = off
restart_after_crash = off
EOF
$PGBIN/pg_controldata -D "$PGDATA" | grep -i "Database block size"
$PGBIN/pg_ctl -D "$PGDATA" -l "$PGDATA/server.log" start -w >/dev/null 2>&1
echo "### 2. build the corrupt index (this is the whole repro)"
$PSQL -v ON_ERROR_STOP=1 <<'EOF'
CREATE EXTENSION amcheck;
CREATE EXTENSION pageinspect;
-- One incompressible 8172-byte value. The resulting index tuple is
-- MAXALIGN(sizeof(IndexTupleData) + VARHDRSZ + 8172) = MAXALIGN(8+4+8172)
-- = 8184 bytes
-- which is the largest index tuple index_form_tuple() will ever produce.
CREATE TABLE pad AS
SELECT substr((SELECT string_agg(md5(random()::text), '')
FROM generate_series(1, 300)), 1, 8172) AS v;
CREATE TABLE t (v text);
-- Must come before CREATE INDEX: the index inherits attstorage, and this
stops
-- TOAST_INDEX_HACK in index_form_tuple() from compressing the value.
ALTER TABLE t ALTER COLUMN v SET STORAGE EXTERNAL;
CREATE INDEX t_v_idx ON t (v);
CHECKPOINT;
-- Four identical keys. Three fill the leaf page; the fourth splits it.
Since
-- all the keys are equal, _bt_truncate() cannot truncate away any key
-- attribute and must append a tiebreaker heap TID: newsize = 8184 + 8 =
8192.
INSERT INTO t SELECT v FROM pad, generate_series(1, 4);
EOF
echo
echo "### 3. amcheck"
$PSQL -c "SELECT bt_index_check('t_v_idx', true);" 2>&1 | sed 's/^/
/'
$PSQL -c "SELECT bt_index_parent_check('t_v_idx', true);" 2>&1 | sed 's/^/
/'
echo "### 4. page layout: zero-length pivot as leaf high key AND as root
downlink"
$PSQL -c "SELECT b AS blkno, s.type, s.live_items, s.avg_item_size
FROM generate_series(1,3) b, bt_page_stats('t_v_idx', b) s;"
$PSQL -c "
WITH g AS (SELECT blk, get_raw_page('t_v_idx', blk) AS pg FROM
generate_series(1,3) blk),
w AS (SELECT blk, off,
get_byte(pg,24+(off-1)*4) + get_byte(pg,25+(off-1)*4)*256
+ get_byte(pg,26+(off-1)*4)*65536 +
get_byte(pg,27+(off-1)*4)*16777216 AS word
FROM g, generate_series(1,3) off)
SELECT blk, off, word & 32767 AS lp_off, (word/32768) & 3 AS lp_flags,
word/131072 AS lp_len
FROM w ORDER BY blk, off;"
echo "### 5. pageinspect underflows on the zero-length item"
$PSQL -Atc "SELECT itemlen FROM bt_page_items('t_v_idx', 1);" 2>&1 | head -2
| sed 's/^/ /'
echo "### 6. silently wrong query results"
echo -n " indexscan: "; $PSQL -Atc \
"SET enable_seqscan=off; SELECT count(*) FROM t WHERE v = (SELECT v FROM
pad);"
echo -n " seqscan: "; $PSQL -Atc \
"SET enable_indexscan=off; SET enable_bitmapscan=off;
SELECT count(*) FROM t WHERE v = (SELECT v FROM pad);"
echo "### 7. the split record is WAL-logged without a full-page image"
$PSQL -Atc "SELECT pg_switch_wal();" >/dev/null
for f in $(ls "$PGDATA/pg_wal" | grep -E '^[0-9A-F]{24}$'); do
$PGBIN/pg_waldump -p "$PGDATA/pg_wal" "$f" 2>/dev/null | grep "SPLIT_R"
done | tail -1 | sed 's/^/ /'
echo "### 8. crash + restart => recovery segfaults, cluster will not come
up"
$PGBIN/pg_ctl -D "$PGDATA" stop -m immediate >/dev/null 2>&1
: > "$PGDATA/server.log"
$PGBIN/pg_ctl -D "$PGDATA" -l "$PGDATA/server.log" start -w -t 20 >/dev/null
2>&1
sed 's/^/ /' "$PGDATA/server.log"
echo "### 9. gdb backtrace of the recovery crash"
if command -v gdb >/dev/null; then
cd "$SOCKDIR" && timeout 180 gdb --batch -q \
-ex 'run' -ex 'bt 4' \
-ex 'p $_siginfo._sifields._sigfault.si_addr' \
--args "$PGBIN/postgres" --single -D "$PGDATA" postgres </dev/null
2>&1 \
| grep -E "SIGSEGV|^#|\\\$1" | sed 's/^/ /'
fi
王飞 <wf13934952799@gmail.com> 于2026年11月3日周二 16:10写道:
Show quoted text
CREATE EXTENSION amcheck;
-- One incompressible 8172-byte value. The resulting index tuple is
-- MAXALIGN(sizeof(IndexTupleData) + VARHDRSZ + 8172) =
MAXALIGN(8+4+8172) = 8184
-- bytes, which is the largest index tuple index_form_tuple() will ever
produce.
CREATE TABLE pad AS
SELECT substr((SELECT string_agg(md5(random()::text), '')
FROM generate_series(1, 300)), 1, 8172) AS v;CREATE TABLE t (v text);
-- Must come before CREATE INDEX: the index inherits attstorage, and this
stops
-- TOAST_INDEX_HACK in index_form_tuple() from compressing the value.
ALTER TABLE t ALTER COLUMN v SET STORAGE EXTERNAL;
CREATE INDEX t_v_idx ON t (v);-- Four identical keys. Three fill the leaf page; the fourth splits it.
Since
-- all keys are equal, _bt_truncate() cannot truncate away any key
attribute and
-- must append a tiebreaker heap TID: newsize = 8184 + 8 = 8192.
INSERT INTO t SELECT v FROM pad, generate_series(1, 4);SELECT bt_index_check('t_v_idx', true);
ERROR: invalid line pointer storage in index "t_v_idx"
DETAIL: Index tid=(3,2) lp_off=32744, lp_len=0 lp_flags=1.PG Bug reporting form <noreply@postgresql.org> 于2026年11月3日周二 15:47写道:
The following bug has been logged on the website:
Bug reference: 19829
Logged by: wangfei
Email address: wf13934952799@gmail.com
PostgreSQL version: 18.6
Operating system: ubuntu-24.04
Description:PostgreSQL version: 19devel (also reproducible on 18.x)
Operating system: Linux x86_64
Configure options: --with-blocksize=32With BLCKSZ=32768, BTMaxItemSize() evaluates to roughly 10896 bytes, but
the
tuple length field reserved by INDEX_SIZE_MASK in IndexTupleData.t_info
can
only represent 8191 bytes. index_form_tuple() enforces the 8191 limit,
so a
leaf tuple is at most 8184 bytes after MAXALIGN. _bt_truncate() may then
need
to append a tiebreaker heap TID:newsize = MAXALIGN(IndexTupleSize(pivot)) +
MAXALIGN(sizeof(ItemPointerData));
...
tidpivot->t_info &= ~INDEX_SIZE_MASK;
tidpivot->t_info |= newsize;For a maximally sized leaf tuple this is 8184 + 8 = 8192, i.e. exactly
INDEX_SIZE_MASK + 1, so the length field wraps to zero. The result is a
zero-length pivot tuple used as a leaf high key; PageAddItem() and
XLogRegisterBufData() both accept it without complaint, so the corruption
is
written out and WAL-logged silently.BTMaxItemSize() is supposed to keep leaf tuples small enough that this
cannot
happen, but the one-third-of-a-page rule scales with BLCKSZ while
INDEX_SIZE_MASK does not, so at BLCKSZ=32768 the one-third rule is no
longer
the binding constraint. Block sizes of 16384 and below are unaffected.Steps to reproduce:
< SQL 脚本>Expected: either the insert is rejected, or the pivot tuple is stored
correctly.
Actual: high key with length 0 (amcheck reports: <具体报错>).A patch against master is attached. It takes Min() of the one-third rule
and
what t_info can represent in BTMaxItemSize(), and adds a defensive
elog(ERROR)
in _bt_truncate() so that a future miscalculation cannot corrupt an index
silently. Since index_form_tuple() already caps index tuples at 8184
bytes,
lowering the BLCKSZ=32768 limit cannot invalidate any index that could
actually exist on disk, so there is no on-disk compatibility concern.