BUG #19595: Three memory-safety defects in src/backend/tsearch/spell.c (dictionary loader), PG 18.3
The following bug has been logged on the website:
Bug reference: 19595
Logged by: Michael Malis
Email address: michaelmalis2@gmail.com
PostgreSQL version: 18.3
Operating system: MacOS
Description:
(I initially filed this at security@ but because the dictionary is
considered a trusted
file Tom asked me to repost here)
Three memory-safety defects in the ispell/hunspell dictionary loader, all
reached by CREATE TEXT SEARCH DICTIONARY on a malformed dictionary file.
BUG 1 -- out-of-bounds heap write in NISortAffixes()
1987: Conf->CompoundAffix = ptr = (CMPDAffix *)
palloc(sizeof(CMPDAffix) * Conf->naffixes);
... /* loop over i < naffixes; ptr++ once per collected affix */
2015: ptr->affix = NULL;
2016: Conf->CompoundAffix = repalloc(Conf->CompoundAffix,
sizeof(CMPDAffix) * (ptr - Conf->CompoundAffix + 1));
The array holds exactly naffixes elements. When every affix is collected,
ptr == base + naffixes, so line 2015 writes one element (8 bytes) past the
end -- and the repalloc that would make room for the terminator is on the
next line, after the write. CMPDAffix is 16 bytes, palloc rounds to
power-of-two chunks, so the write escapes its chunk when naffixes is a power
of two, and always escapes for naffixes > 512 (dedicated block), which
covers typical dictionaries. The store then lands in the next chunk's header
or in malloc metadata.
Reproducer -- in $SHAREDIR/tsearch_data, oob.affix:
compoundwords controlled Z
suffixes
flag ~Z:
. > S
oob.dict:
foo/Z
then:
CREATE TEXT SEARCH DICTIONARY oob (TEMPLATE = ispell, DictFile =
oob, AffFile = oob);
SELECT ts_lexize('oob', 'foos');
This gives naffixes == 1 with the one affix collected. The allocator detects
the damage at the next allocation ("free list is damaged", aborting in
palloc from mkANode from NISortAffixes).
Fix: allocate naffixes + 1 at line 1987, or write the terminator after the
repalloc.
BUG 2 -- uninitialized stack buffer read in NIImportAffixes()
1426: char flag[BUFSIZ]; /* never initialized */
...
1519: flag[0] = *s++; /* only written in the "flag" branch
*/
1520: flag[1] = '\0';
...
1543: NIAddAffix(Conf, flag, flagflags, mask, find, repl, ...);
/* unconditional */
flag is written only inside the "flag" directive branch but passed
unconditionally to NIAddAffix, which does cpstrdup(Conf, flag) -- strlen +
strcpy over uninitialized stack. If no "flag" line was parsed, this is an
unbounded strlen (no guaranteed NUL in BUFSIZ) and stack contents are copied
into a long-lived dictionary flag.
Trigger: a .affix file with an old-format "prefixes"/"suffixes" section and
a parseable affix entry but no "flag" directive, e.g.:
COMPOUNDWORDS l 1
suffixes
nlag Z:
. > S
with dict "foo/Z" ("nlag" is simply not "flag", so the directive is never
seen while the entry still parses).
Fix: initialize flag[0] = '\0' at declaration.
BUG 3 -- NULL dereference on unfilled AF alias slots
1324: Conf->AffixData = (const char **) palloc0(naffix * sizeof(char *));
1336: Conf->AffixData[curaffix] = cpstrdup(Conf, sflag); /* one
per AF line */
The alias table is zero-filled and only the AF lines actually present are
filled. If "AF <n>" declares more slots than the file fills, the tail stays
NULL, and those NULL slots are dereferenced without a check:
- MergeAffix() line 1576: if (*Conf->AffixData[a1] == '\0')
(the Assert at 1573 checks only the index, and asserts are off in
production builds), reached from NISortDictionary() at line 1691;
- getAffixFlagSet() (1156) -> getCompoundAffixFlagValue() (1120) ->
getNextFlagFromString() (350), reached from inside NIImportOOAffixes().
Trigger: a .affix file whose AF table declares a larger count than the
number of AF lines it provides, with a dictionary word referencing an
unfilled index.
Fix: reject an incompletely populated AF table, or treat a NULL slot as the
empty flag set (VoidString) at the dereference sites.
Reachability: CREATE TEXT SEARCH DICTIONARY requires CREATE on a schema, not
superuser (DefineTSDictionary in src/backend/commands/tsearchcmds.c). The
file path is restricted to [a-z0-9_] under $SHAREDIR/tsearch_data
(get_tsearch_config_filename in src/backend/tsearch/ts_utils.c), so the
crafted file must be placed there by other means -- most realistically a
corrupt or hostile third-party hunspell dictionary. Minimized file pairs
available on request.
Hi, Michael!
Patch attached.
1. NISortAffixes() — allocate CompoundAffix with naffixes + 1 so the
terminating NULL entry fits. The previous allocation of exactly
naffixes elements left the terminator one past the end when every
affix was collected.
2. NIImportAffixes() — initialize flag[0] = '\0' so NIAddAffix() does
not see an uninitialized buffer when an old-format affix entry
appears without a preceding "flag" directive.
3. Incomplete Hunspell AF tables — reject a NULL AffixData slot in
getAffixFlagSet() as an invalid affix alias (same error text as an
out-of-range alias), and reject an incompletely populated AF table
at the end of NIImportOOAffixes(), mirroring the existing "too many
aliases" check. The former covers a mid-parse reference to an
unfilled slot, the latter covers a truncated AF table with no such
reference.
Regression tests cover both AF cases. The former errors while
tsearch_readline() still has an error-context callback installed, so
the ERROR would otherwise include CONTEXT with the absolute path of
the affix file under $SHAREDIR. That path is install-dependent and
would make the expected file non-portable, so the test wraps that
CREATE in \set VERBOSITY terse (same pattern as elsewhere in the
regress suite). The incomplete-table case errors after
tsearch_readline_end(), so it needs no VERBOSITY tweak.
вс, 2 авг. 2026 г. в 08:30, PG Bug reporting form <noreply@postgresql.org>:
Show quoted text
The following bug has been logged on the website:
Bug reference: 19595
Logged by: Michael Malis
Email address: michaelmalis2@gmail.com
PostgreSQL version: 18.3
Operating system: MacOS
Description:(I initially filed this at security@ but because the dictionary is
considered a trusted
file Tom asked me to repost here)Three memory-safety defects in the ispell/hunspell dictionary loader, all
reached by CREATE TEXT SEARCH DICTIONARY on a malformed dictionary file.BUG 1 -- out-of-bounds heap write in NISortAffixes()
1987: Conf->CompoundAffix = ptr = (CMPDAffix *)
palloc(sizeof(CMPDAffix) * Conf->naffixes);
... /* loop over i < naffixes; ptr++ once per collected affix */
2015: ptr->affix = NULL;
2016: Conf->CompoundAffix = repalloc(Conf->CompoundAffix,
sizeof(CMPDAffix) * (ptr - Conf->CompoundAffix + 1));The array holds exactly naffixes elements. When every affix is collected,
ptr == base + naffixes, so line 2015 writes one element (8 bytes) past the
end -- and the repalloc that would make room for the terminator is on the
next line, after the write. CMPDAffix is 16 bytes, palloc rounds to
power-of-two chunks, so the write escapes its chunk when naffixes is a
power
of two, and always escapes for naffixes > 512 (dedicated block), which
covers typical dictionaries. The store then lands in the next chunk's
header
or in malloc metadata.Reproducer -- in $SHAREDIR/tsearch_data, oob.affix:
compoundwords controlled Z
suffixes
flag ~Z:
. > Soob.dict:
foo/Z
then:
CREATE TEXT SEARCH DICTIONARY oob (TEMPLATE = ispell, DictFile =
oob, AffFile = oob);
SELECT ts_lexize('oob', 'foos');This gives naffixes == 1 with the one affix collected. The allocator
detects
the damage at the next allocation ("free list is damaged", aborting in
palloc from mkANode from NISortAffixes).Fix: allocate naffixes + 1 at line 1987, or write the terminator after the
repalloc.BUG 2 -- uninitialized stack buffer read in NIImportAffixes()
1426: char flag[BUFSIZ]; /* never initialized */
...
1519: flag[0] = *s++; /* only written in the "flag"
branch
*/
1520: flag[1] = '\0';
...
1543: NIAddAffix(Conf, flag, flagflags, mask, find, repl, ...);
/* unconditional */flag is written only inside the "flag" directive branch but passed
unconditionally to NIAddAffix, which does cpstrdup(Conf, flag) -- strlen +
strcpy over uninitialized stack. If no "flag" line was parsed, this is an
unbounded strlen (no guaranteed NUL in BUFSIZ) and stack contents are
copied
into a long-lived dictionary flag.Trigger: a .affix file with an old-format "prefixes"/"suffixes" section and
a parseable affix entry but no "flag" directive, e.g.:COMPOUNDWORDS l 1
suffixes
nlag Z:
. > Swith dict "foo/Z" ("nlag" is simply not "flag", so the directive is never
seen while the entry still parses).Fix: initialize flag[0] = '\0' at declaration.
BUG 3 -- NULL dereference on unfilled AF alias slots
1324: Conf->AffixData = (const char **) palloc0(naffix * sizeof(char
*));
1336: Conf->AffixData[curaffix] = cpstrdup(Conf, sflag); /* one
per AF line */The alias table is zero-filled and only the AF lines actually present are
filled. If "AF <n>" declares more slots than the file fills, the tail stays
NULL, and those NULL slots are dereferenced without a check:- MergeAffix() line 1576: if (*Conf->AffixData[a1] == '\0')
(the Assert at 1573 checks only the index, and asserts are off in
production builds), reached from NISortDictionary() at line 1691;
- getAffixFlagSet() (1156) -> getCompoundAffixFlagValue() (1120) ->
getNextFlagFromString() (350), reached from inside NIImportOOAffixes().Trigger: a .affix file whose AF table declares a larger count than the
number of AF lines it provides, with a dictionary word referencing an
unfilled index.Fix: reject an incompletely populated AF table, or treat a NULL slot as the
empty flag set (VoidString) at the dereference sites.Reachability: CREATE TEXT SEARCH DICTIONARY requires CREATE on a schema,
not
superuser (DefineTSDictionary in src/backend/commands/tsearchcmds.c). The
file path is restricted to [a-z0-9_] under $SHAREDIR/tsearch_data
(get_tsearch_config_filename in src/backend/tsearch/ts_utils.c), so the
crafted file must be placed there by other means -- most realistically a
corrupt or hostile third-party hunspell dictionary. Minimized file pairs
available on request.
Andrey Rachitskiy <pl0h0yp1@gmail.com> writes:
Hi, Michael!
Patch attached.
I pushed these code changes with one minor tweak: adjusting the
new error message in NIImportOOAffixes to look more like the
existing one about too many aliases.
I left out the test cases. I don't think we need them, and
I certainly don't think we want to install intentionally-broken
files as sample data, as this patch would have done.
regards, tom lane
Many thanks to Tom for merging the patch and for the review, to Michael for
the report, and I'm very glad to be of help!
вс, 2 авг. 2026 г. в 22:25, Tom Lane <tgl@sss.pgh.pa.us>:
Andrey Rachitskiy <pl0h0yp1@gmail.com> writes:
Hi, Michael!
Patch attached.I pushed these code changes with one minor tweak: adjusting the
new error message in NIImportOOAffixes to look more like the
existing one about too many aliases.I left out the test cases. I don't think we need them, and
I certainly don't think we want to install intentionally-broken
files as sample data, as this patch would have done.regards, tom lane
--
Regards,
Rachitskiy Andrey
Hello Tom,
02.08.2026 20:25, Tom Lane wrote:
Andrey Rachitskiy<pl0h0yp1@gmail.com> writes:
Hi, Michael!
Patch attached.I pushed these code changes with one minor tweak: adjusting the
new error message in NIImportOOAffixes to look more like the
existing one about too many aliases.I left out the test cases. I don't think we need them, and
I certainly don't think we want to install intentionally-broken
files as sample data, as this patch would have done.
I'm not sure it's directly related to this bug report, but maybe you'd
like to fix one more memory-safety defect in tsearch in passing...
With the oom-simulation patch applied, the following script:
for i in {1..10}; do
echo "
SELECT COUNT(*) FROM pg_ts_dict;
CREATE TEXT SEARCH DICTIONARY thesaurus (Template=thesaurus, DictFile=thesaurus_sample, Dictionary=english_stem);
CREATE TEXT SEARCH CONFIGURATION tst (COPY=english);
SELECT to_tsvector('tst', 'Test test');
DROP TEXT SEARCH CONFIGURATION tst;
DROP TEXT SEARCH DICTIONARY thesaurus;
" | psql
grep 'was terminated' server.log && break;
done
fails for me as below:
2026-08-02 19:48:45.625 UTC [560023] LOG: client backend (PID 560036) was terminated by signal 11: Segmentation fault
Core was generated by `postgres: law regression [local] SELECT '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000055aecffd519e in MemoryContextSetIdentifier (context=0x7f7f7f7f7f7f7f7f, id=0x0) at mcxt.c:667
667 Assert(MemoryContextIsValid(context));
(gdb) bt
#0 0x000055aecffd519e in MemoryContextSetIdentifier (context=0x7f7f7f7f7f7f7f7f, id=0x0) at mcxt.c:667
#1 0x000055aecff863c1 in lookup_ts_dictionary_cache (dictId=13336) at ts_cache.c:307
#2 0x000055aecfd87f00 in LexizeExec (ld=0x7ffe75617c20, correspondLexem=0x0) at ts_parse.c:204
#3 0x000055aecfd88741 in parsetext (cfgId=16385, prs=0x7ffe75617cc0, buf=0x55aeee42bba4 "Test test~\177\1770",
buflen=9) at ts_parse.c:402
#4 0x000055aecfd8667d in to_tsvector_byid (fcinfo=0x55aeee5188c0) at to_tsany.c:260
#5 0x000055aecfa2a399 in ExecInterpExpr (state=0x55aeee5187e0, econtext=0x55aeee518d20, isnull=0x7ffe75618064) at
execExprInterp.c:1011
#6 0x000055aecfa2cf0b in ExecInterpExprStillValid (state=0x55aeee5187e0, econtext=0x55aeee518d20,
isNull=0x7ffe75618064) at execExprInterp.c:2309
#7 0x000055aecfbfba26 in ExecEvalExprSwitchContext (state=0x55aeee5187e0, econtext=0x55aeee518d20, isNull=0x7ffe75618064)
at ../../../../src/include/executor/executor.h:452
Plain `make check` triggers similar crashes as well...
Best regards,
Alexander
Attachments:
lookup_ts_dictionary_cache-oom.patchtext/x-patch; charset=UTF-8; name=lookup_ts_dictionary_cache-oom.patchDownload+6-1
Alexander Lakhin <exclusion@gmail.com> writes:
I'm not sure it's directly related to this bug report, but maybe you'd
like to fix one more memory-safety defect in tsearch in passing...
Hmph. Not sure I'd call that "memory safety", but yeah, this bit
isn't being careful about having a valid intermediate state of the
data structure. Thanks for the report!
regards, tom lane