Fixing out-of-range warnings with late-model gcc+UBSAN
Buildfarm member thorntail was recently resurrected with a very
recent gcc (Debian 15.2.0-9), and since then it's been giving
these compiler warnings:
In file included from ../../../../src/include/storage/bufmgr.h:21,
from ../../../../src/include/access/bufmask.h:21,
from heapam_xlog.c:17:
In function 'PageGetItemId',
inlined from 'heap_xlog_update' at heapam_xlog.c:885:9:
../../../../src/include/storage/bufpage.h:246:16: warning: array subscript -1 is below array bounds of 'ItemIdData[]' [-Warray-bounds=]
246 | return &((PageHeader) page)->pd_linp[offsetNumber - 1];
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../../src/include/storage/bufpage.h: In function 'heap_xlog_update':
../../../../src/include/storage/bufpage.h:170:25: note: while referencing 'pd_linp'
170 | ItemIdData pd_linp[FLEXIBLE_ARRAY_MEMBER]; /* line pointer array */
| ^~~~~~~
In function 'PageGetItemId',
inlined from 'heap_xlog_delete' at heapam_xlog.c:455:9,
inlined from 'heap_redo' at heapam_xlog.c:1321:4:
../../../../src/include/storage/bufpage.h:246:16: warning: array subscript -1 is below array bounds of 'ItemIdData[]' [-Warray-bounds=]
246 | return &((PageHeader) page)->pd_linp[offsetNumber - 1];
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../../src/include/storage/bufpage.h: In function 'heap_redo':
../../../../src/include/storage/bufpage.h:170:25: note: while referencing 'pd_linp'
170 | ItemIdData pd_linp[FLEXIBLE_ARRAY_MEMBER]; /* line pointer array */
| ^~~~~~~
In file included from ../../../../src/include/access/itup.h:19,
from ../../../../src/include/access/nbtree.h:18,
from nbtsearch.c:18:
In function 'PageGetItemId',
inlined from '_bt_get_endpoint' at nbtsearch.c:2151:41:
../../../../src/include/storage/bufpage.h:246:16: warning: array subscript -1 is below array bounds of 'ItemIdData[]' [-Warray-bounds=]
246 | return &((PageHeader) page)->pd_linp[offsetNumber - 1];
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../../src/include/storage/bufpage.h: In function '_bt_get_endpoint':
../../../../src/include/storage/bufpage.h:170:25: note: while referencing 'pd_linp'
170 | ItemIdData pd_linp[FLEXIBLE_ARRAY_MEMBER]; /* line pointer array */
| ^~~~~~~
I've been able to replicate this locally using Fedora rawhide's
compiler (Red Hat 15.2.1-4) and thorntail's build settings
'CC' => 'ccache gcc -fsanitize=undefined -fsanitize-undefined-trap-on-error',
'CFLAGS' => '-O2 -funwind-tables',
So it seems like we ought to do something about this before more
platforms start complaining. The problem evidently is that the
compiler can't satisfy itself that offsetNumber > 0. These call
sites do mostly have range checks on the offset, but they're
one-sided, checking only for offset too large.
Looking at the instances in heapam_xlog.c, they are very
badly coded anyway IMO, having duplicate range checks and
using the same elog message for two distinguishable problems.
So I propose the attached patch, which cleans up all copies
of that coding pattern (in this file anyway) even though
right now only two are producing warnings.
regards, tom lane