BUG #19595: Three memory-safety defects in src/backend/tsearch/spell.c (dictionary loader), PG 18.3
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t253280psql -h localhost -U postgresBuilt from patchset v13 (message #13), October 22, 2026 at 02:49 PM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t253280_13 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t253280_13 && git checkout t253280_13Patchset v13 (message #13) is on t253280_13
The following bug has been logged on the website:
Bug reference: 19595
Logged by: Michael Malis
Email address: michaelmalis2@gmail.com
PostgreSQL version: 18.3
Operating system: MacOS
Description:
(I initially filed this at security@ but because the dictionary is
considered a trusted
file Tom asked me to repost here)
Three memory-safety defects in the ispell/hunspell dictionary loader, all
reached by CREATE TEXT SEARCH DICTIONARY on a malformed dictionary file.
BUG 1 -- out-of-bounds heap write in NISortAffixes()
1987: Conf->CompoundAffix = ptr = (CMPDAffix *)
palloc(sizeof(CMPDAffix) * Conf->naffixes);
... /* loop over i < naffixes; ptr++ once per collected affix */
2015: ptr->affix = NULL;
2016: Conf->CompoundAffix = repalloc(Conf->CompoundAffix,
sizeof(CMPDAffix) * (ptr - Conf->CompoundAffix + 1));
The array holds exactly naffixes elements. When every affix is collected,
ptr == base + naffixes, so line 2015 writes one element (8 bytes) past the
end -- and the repalloc that would make room for the terminator is on the
next line, after the write. CMPDAffix is 16 bytes, palloc rounds to
power-of-two chunks, so the write escapes its chunk when naffixes is a power
of two, and always escapes for naffixes > 512 (dedicated block), which
covers typical dictionaries. The store then lands in the next chunk's header
or in malloc metadata.
Reproducer -- in $SHAREDIR/tsearch_data, oob.affix:
compoundwords controlled Z
suffixes
flag ~Z:
. > S
oob.dict:
foo/Z
then:
CREATE TEXT SEARCH DICTIONARY oob (TEMPLATE = ispell, DictFile =
oob, AffFile = oob);
SELECT ts_lexize('oob', 'foos');
This gives naffixes == 1 with the one affix collected. The allocator detects
the damage at the next allocation ("free list is damaged", aborting in
palloc from mkANode from NISortAffixes).
Fix: allocate naffixes + 1 at line 1987, or write the terminator after the
repalloc.
BUG 2 -- uninitialized stack buffer read in NIImportAffixes()
1426: char flag[BUFSIZ]; /* never initialized */
...
1519: flag[0] = *s++; /* only written in the "flag" branch
*/
1520: flag[1] = '\0';
...
1543: NIAddAffix(Conf, flag, flagflags, mask, find, repl, ...);
/* unconditional */
flag is written only inside the "flag" directive branch but passed
unconditionally to NIAddAffix, which does cpstrdup(Conf, flag) -- strlen +
strcpy over uninitialized stack. If no "flag" line was parsed, this is an
unbounded strlen (no guaranteed NUL in BUFSIZ) and stack contents are copied
into a long-lived dictionary flag.
Trigger: a .affix file with an old-format "prefixes"/"suffixes" section and
a parseable affix entry but no "flag" directive, e.g.:
COMPOUNDWORDS l 1
suffixes
nlag Z:
. > S
with dict "foo/Z" ("nlag" is simply not "flag", so the directive is never
seen while the entry still parses).
Fix: initialize flag[0] = '\0' at declaration.
BUG 3 -- NULL dereference on unfilled AF alias slots
1324: Conf->AffixData = (const char **) palloc0(naffix * sizeof(char *));
1336: Conf->AffixData[curaffix] = cpstrdup(Conf, sflag); /* one
per AF line */
The alias table is zero-filled and only the AF lines actually present are
filled. If "AF <n>" declares more slots than the file fills, the tail stays
NULL, and those NULL slots are dereferenced without a check:
- MergeAffix() line 1576: if (*Conf->AffixData[a1] == '\0')
(the Assert at 1573 checks only the index, and asserts are off in
production builds), reached from NISortDictionary() at line 1691;
- getAffixFlagSet() (1156) -> getCompoundAffixFlagValue() (1120) ->
getNextFlagFromString() (350), reached from inside NIImportOOAffixes().
Trigger: a .affix file whose AF table declares a larger count than the
number of AF lines it provides, with a dictionary word referencing an
unfilled index.
Fix: reject an incompletely populated AF table, or treat a NULL slot as the
empty flag set (VoidString) at the dereference sites.
Reachability: CREATE TEXT SEARCH DICTIONARY requires CREATE on a schema, not
superuser (DefineTSDictionary in src/backend/commands/tsearchcmds.c). The
file path is restricted to [a-z0-9_] under $SHAREDIR/tsearch_data
(get_tsearch_config_filename in src/backend/tsearch/ts_utils.c), so the
crafted file must be placed there by other means -- most realistically a
corrupt or hostile third-party hunspell dictionary. Minimized file pairs
available on request.
Hi, Michael!
Patch attached.
1. NISortAffixes() — allocate CompoundAffix with naffixes + 1 so the
terminating NULL entry fits. The previous allocation of exactly
naffixes elements left the terminator one past the end when every
affix was collected.
2. NIImportAffixes() — initialize flag[0] = '\0' so NIAddAffix() does
not see an uninitialized buffer when an old-format affix entry
appears without a preceding "flag" directive.
3. Incomplete Hunspell AF tables — reject a NULL AffixData slot in
getAffixFlagSet() as an invalid affix alias (same error text as an
out-of-range alias), and reject an incompletely populated AF table
at the end of NIImportOOAffixes(), mirroring the existing "too many
aliases" check. The former covers a mid-parse reference to an
unfilled slot, the latter covers a truncated AF table with no such
reference.
Regression tests cover both AF cases. The former errors while
tsearch_readline() still has an error-context callback installed, so
the ERROR would otherwise include CONTEXT with the absolute path of
the affix file under $SHAREDIR. That path is install-dependent and
would make the expected file non-portable, so the test wraps that
CREATE in \set VERBOSITY terse (same pattern as elsewhere in the
regress suite). The incomplete-table case errors after
tsearch_readline_end(), so it needs no VERBOSITY tweak.
вс, 2 авг. 2026 г. в 08:30, PG Bug reporting form <noreply@postgresql.org>:
Show quoted text
The following bug has been logged on the website:
Bug reference: 19595
Logged by: Michael Malis
Email address: michaelmalis2@gmail.com
PostgreSQL version: 18.3
Operating system: MacOS
Description:(I initially filed this at security@ but because the dictionary is
considered a trusted
file Tom asked me to repost here)Three memory-safety defects in the ispell/hunspell dictionary loader, all
reached by CREATE TEXT SEARCH DICTIONARY on a malformed dictionary file.BUG 1 -- out-of-bounds heap write in NISortAffixes()
1987: Conf->CompoundAffix = ptr = (CMPDAffix *)
palloc(sizeof(CMPDAffix) * Conf->naffixes);
... /* loop over i < naffixes; ptr++ once per collected affix */
2015: ptr->affix = NULL;
2016: Conf->CompoundAffix = repalloc(Conf->CompoundAffix,
sizeof(CMPDAffix) * (ptr - Conf->CompoundAffix + 1));The array holds exactly naffixes elements. When every affix is collected,
ptr == base + naffixes, so line 2015 writes one element (8 bytes) past the
end -- and the repalloc that would make room for the terminator is on the
next line, after the write. CMPDAffix is 16 bytes, palloc rounds to
power-of-two chunks, so the write escapes its chunk when naffixes is a
power
of two, and always escapes for naffixes > 512 (dedicated block), which
covers typical dictionaries. The store then lands in the next chunk's
header
or in malloc metadata.Reproducer -- in $SHAREDIR/tsearch_data, oob.affix:
compoundwords controlled Z
suffixes
flag ~Z:
. > Soob.dict:
foo/Z
then:
CREATE TEXT SEARCH DICTIONARY oob (TEMPLATE = ispell, DictFile =
oob, AffFile = oob);
SELECT ts_lexize('oob', 'foos');This gives naffixes == 1 with the one affix collected. The allocator
detects
the damage at the next allocation ("free list is damaged", aborting in
palloc from mkANode from NISortAffixes).Fix: allocate naffixes + 1 at line 1987, or write the terminator after the
repalloc.BUG 2 -- uninitialized stack buffer read in NIImportAffixes()
1426: char flag[BUFSIZ]; /* never initialized */
...
1519: flag[0] = *s++; /* only written in the "flag"
branch
*/
1520: flag[1] = '\0';
...
1543: NIAddAffix(Conf, flag, flagflags, mask, find, repl, ...);
/* unconditional */flag is written only inside the "flag" directive branch but passed
unconditionally to NIAddAffix, which does cpstrdup(Conf, flag) -- strlen +
strcpy over uninitialized stack. If no "flag" line was parsed, this is an
unbounded strlen (no guaranteed NUL in BUFSIZ) and stack contents are
copied
into a long-lived dictionary flag.Trigger: a .affix file with an old-format "prefixes"/"suffixes" section and
a parseable affix entry but no "flag" directive, e.g.:COMPOUNDWORDS l 1
suffixes
nlag Z:
. > Swith dict "foo/Z" ("nlag" is simply not "flag", so the directive is never
seen while the entry still parses).Fix: initialize flag[0] = '\0' at declaration.
BUG 3 -- NULL dereference on unfilled AF alias slots
1324: Conf->AffixData = (const char **) palloc0(naffix * sizeof(char
*));
1336: Conf->AffixData[curaffix] = cpstrdup(Conf, sflag); /* one
per AF line */The alias table is zero-filled and only the AF lines actually present are
filled. If "AF <n>" declares more slots than the file fills, the tail stays
NULL, and those NULL slots are dereferenced without a check:- MergeAffix() line 1576: if (*Conf->AffixData[a1] == '\0')
(the Assert at 1573 checks only the index, and asserts are off in
production builds), reached from NISortDictionary() at line 1691;
- getAffixFlagSet() (1156) -> getCompoundAffixFlagValue() (1120) ->
getNextFlagFromString() (350), reached from inside NIImportOOAffixes().Trigger: a .affix file whose AF table declares a larger count than the
number of AF lines it provides, with a dictionary word referencing an
unfilled index.Fix: reject an incompletely populated AF table, or treat a NULL slot as the
empty flag set (VoidString) at the dereference sites.Reachability: CREATE TEXT SEARCH DICTIONARY requires CREATE on a schema,
not
superuser (DefineTSDictionary in src/backend/commands/tsearchcmds.c). The
file path is restricted to [a-z0-9_] under $SHAREDIR/tsearch_data
(get_tsearch_config_filename in src/backend/tsearch/ts_utils.c), so the
crafted file must be placed there by other means -- most realistically a
corrupt or hostile third-party hunspell dictionary. Minimized file pairs
available on request.
Andrey Rachitskiy <pl0h0yp1@gmail.com> writes:
Hi, Michael!
Patch attached.
I pushed these code changes with one minor tweak: adjusting the
new error message in NIImportOOAffixes to look more like the
existing one about too many aliases.
I left out the test cases. I don't think we need them, and
I certainly don't think we want to install intentionally-broken
files as sample data, as this patch would have done.
regards, tom lane
Many thanks to Tom for merging the patch and for the review, to Michael for
the report, and I'm very glad to be of help!
вс, 2 авг. 2026 г. в 22:25, Tom Lane <tgl@sss.pgh.pa.us>:
Andrey Rachitskiy <pl0h0yp1@gmail.com> writes:
Hi, Michael!
Patch attached.I pushed these code changes with one minor tweak: adjusting the
new error message in NIImportOOAffixes to look more like the
existing one about too many aliases.I left out the test cases. I don't think we need them, and
I certainly don't think we want to install intentionally-broken
files as sample data, as this patch would have done.regards, tom lane
--
Regards,
Rachitskiy Andrey
Hello Tom,
02.08.2026 20:25, Tom Lane wrote:
Andrey Rachitskiy<pl0h0yp1@gmail.com> writes:
Hi, Michael!
Patch attached.I pushed these code changes with one minor tweak: adjusting the
new error message in NIImportOOAffixes to look more like the
existing one about too many aliases.I left out the test cases. I don't think we need them, and
I certainly don't think we want to install intentionally-broken
files as sample data, as this patch would have done.
I'm not sure it's directly related to this bug report, but maybe you'd
like to fix one more memory-safety defect in tsearch in passing...
With the oom-simulation patch applied, the following script:
for i in {1..10}; do
echo "
SELECT COUNT(*) FROM pg_ts_dict;
CREATE TEXT SEARCH DICTIONARY thesaurus (Template=thesaurus, DictFile=thesaurus_sample, Dictionary=english_stem);
CREATE TEXT SEARCH CONFIGURATION tst (COPY=english);
SELECT to_tsvector('tst', 'Test test');
DROP TEXT SEARCH CONFIGURATION tst;
DROP TEXT SEARCH DICTIONARY thesaurus;
" | psql
grep 'was terminated' server.log && break;
done
fails for me as below:
2026-08-02 19:48:45.625 UTC [560023] LOG: client backend (PID 560036) was terminated by signal 11: Segmentation fault
Core was generated by `postgres: law regression [local] SELECT '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000055aecffd519e in MemoryContextSetIdentifier (context=0x7f7f7f7f7f7f7f7f, id=0x0) at mcxt.c:667
667 Assert(MemoryContextIsValid(context));
(gdb) bt
#0 0x000055aecffd519e in MemoryContextSetIdentifier (context=0x7f7f7f7f7f7f7f7f, id=0x0) at mcxt.c:667
#1 0x000055aecff863c1 in lookup_ts_dictionary_cache (dictId=13336) at ts_cache.c:307
#2 0x000055aecfd87f00 in LexizeExec (ld=0x7ffe75617c20, correspondLexem=0x0) at ts_parse.c:204
#3 0x000055aecfd88741 in parsetext (cfgId=16385, prs=0x7ffe75617cc0, buf=0x55aeee42bba4 "Test test~\177\1770",
buflen=9) at ts_parse.c:402
#4 0x000055aecfd8667d in to_tsvector_byid (fcinfo=0x55aeee5188c0) at to_tsany.c:260
#5 0x000055aecfa2a399 in ExecInterpExpr (state=0x55aeee5187e0, econtext=0x55aeee518d20, isnull=0x7ffe75618064) at
execExprInterp.c:1011
#6 0x000055aecfa2cf0b in ExecInterpExprStillValid (state=0x55aeee5187e0, econtext=0x55aeee518d20,
isNull=0x7ffe75618064) at execExprInterp.c:2309
#7 0x000055aecfbfba26 in ExecEvalExprSwitchContext (state=0x55aeee5187e0, econtext=0x55aeee518d20, isNull=0x7ffe75618064)
at ../../../../src/include/executor/executor.h:452
Plain `make check` triggers similar crashes as well...
Best regards,
Alexander
Attachments:
lookup_ts_dictionary_cache-oom.patchtext/x-patch; charset=UTF-8; name=lookup_ts_dictionary_cache-oom.patchDownload+6-1
Alexander Lakhin <exclusion@gmail.com> writes:
I'm not sure it's directly related to this bug report, but maybe you'd
like to fix one more memory-safety defect in tsearch in passing...
Hmph. Not sure I'd call that "memory safety", but yeah, this bit
isn't being careful about having a valid intermediate state of the
data structure. Thanks for the report!
regards, tom lane
Hi,
One more problem in the same file. It is not one of the three in the
original report - those were all on the affix-rule side (CompoundAffix,
the flag buffer, the AF alias table), while this one is in the compound
flag table - and it is older than all of them, the code being from 9.6.
So I'm posting here rather than opening a new report.
CompoundAffixFlag holds a flag in a union whose member is chosen by the
flag mode the affix file's FLAG line declares, and NIImportOOAffixes()
converts each COMPOUNDFLAG / ONLYINCOMPOUND / ... flag as soon as it reads
the line, using the mode in effect at that point. FLAG may appear anywhere
in the file, so flags read before and after it can disagree about which
member holds the value. cmpcmdflag() takes the mode from its first
argument alone and applies it to both, so it can read an integer as a char
pointer and hand it to strcmp().
Three ways this shows up. Starting with the one that doesn't crash, which
I think matters most: take the shipped hunspell_sample_num dictionary and
move its FLAG line below the compound options, changing nothing else.
COMPOUNDFLAG 101
ONLYINCOMPOUND 102
FLAG num
SELECT ts_lexize('withflagfirst', 'footballklubber');
{footballklubber,foot,ball,klubber,football,klubber}
SELECT ts_lexize('withflaglast', 'footballklubber');
{footballklubber}
The entries are stored as strings while the lookup key is built with the
final mode, so the numeric comparison compares a pointer's low half against
a number, never matches, and the compound flag is never found. No error,
no warning, and suffix handling still works ('books' -> {book}), so the
dictionary looks healthy. Compound splitting is simply gone, along with
every lexeme it would have produced: a search for 'ball' stops finding the
document, and a GIN index built this way never contained those lexemes.
The two crashing shapes:
COMPOUNDFLAG A / FLAG num / COMPOUNDBEGIN 1
-> heterogeneous array, dies in the sort
SIGSEGV __strcmp_evex <- cmpcmdflag spell.c:226 <- pg_qsort
<- NIImportOOAffixes spell.c:1306
FLAG num / COMPOUNDFLAG 101 / ONLYINCOMPOUND 102 / FLAG long
-> array consistent, but the lookup key uses the final mode
SIGSEGV <- cmpcmdflag spell.c:226 <- bsearch
<- getCompoundAffixFlagValue spell.c:1152
<- makeCompoundFlags spell.c:1652 <- mkSPNode spell.c:1725
With --enable-cassert the Assert() in cmpcmdflag() fires first. All three
reproduce on 8646214ee20 on both build types, and the code involved has not
changed in a long time. Reachability is as for the defects fixed by
330a72052cd - you must be able to place a file in $SHAREDIR/tsearch_data -
and every dictionary I know of, including all four shipped samples, puts
FLAG first, which is presumably why this went unreported for ten years.
0001 keeps the flags as strings while the file is read and converts them
once it has been read in full, when the mode is final. That removes all
three symptoms, since the array and the lookup key then use the same
representation.
It also makes the position of FLAG irrelevant, which is what the rest of
the parser already does: AF, SFX and PFX flags are parsed in a second pass
and so always use the final mode. PG is in fact already more permissive
than hunspell(5), which says "If the affix file contains the FLAG parameter,
define it before the AF definitions" - move FLAG below the AF lines of
hunspell_sample_long and PG still gets it right. So FLAG is currently
file-scoped for AF/SFX/PFX and line-scoped for the eight compound options;
0001 makes the latter agree. Erroring out instead would be about six
lines, but that would leave us stricter than Hunspell for compound options
while staying looser for AF, which seems hard to justify.
Note the old ispell format reaches this code from NIImportAffixes() and
returns without entering NIImportOOAffixes(), so it needs the conversion
too - I missed that at first and ispell_sample promptly crashed the
regression run.
0002 is optional cleanup. With 0001 in place every entry agrees
with Conf->flagMode, so the per-entry copy of the mode is redundant; it
exists only because of the comment on the field, "we don't have a
bsearch_arg version, so, copy FlagMode", which stopped being true when
bsearch_arg() moved to src/port in bfa2cee7841. Taking the mode through
qsort_arg()/bsearch_arg() lets the field and the Assert() go. No functional
change, and 0001 does not depend on it.
Regards,
Ewan Young
On Mon, Aug 3, 2026 at 4:11 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Alexander Lakhin <exclusion@gmail.com> writes:
I'm not sure it's directly related to this bug report, but maybe you'd
like to fix one more memory-safety defect in tsearch in passing...Hmph. Not sure I'd call that "memory safety", but yeah, this bit
isn't being careful about having a valid intermediate state of the
data structure. Thanks for the report!regards, tom lane
--
Regards,
Ewan Young
Attachments:
t253280_7v1-0001-Don-t-convert-Hunspell-compound-flags-before-the-fla.patchapplication/octet-stream; name=v1-0001-Don-t-convert-Hunspell-compound-flags-before-the-fla.patchDownload+80-20
v1-0002-Drop-the-per-entry-copy-of-the-flag-mode-in-Compound.patchapplication/octet-stream; name=v1-0002-Drop-the-per-entry-copy-of-the-flag-mode-in-Compound.patchDownload+11-18
Ewan Young <kdbase.hack@gmail.com> writes:
One more problem in the same file. It is not one of the three in the
original report - those were all on the affix-rule side (CompoundAffix,
the flag buffer, the AF alias table), while this one is in the compound
flag table - and it is older than all of them, the code being from 9.6.
So I'm posting here rather than opening a new report.
Pushed, thanks for the report!
I noticed while reading your patch that all of the strtol() calls in
this file store the result into an "int" not a "long", which opens
the door to an undetected integer overflow and truncation. Nothing
terribly harmful seems likely to ensue, but I thought I'd clean that
up too while we're here.
regards, tom lane
Hello Tom,
02.08.2026 23:11, Tom Lane wrote:
Hmph. Not sure I'd call that "memory safety", but yeah, this bit
isn't being careful about having a valid intermediate state of the
data structure. Thanks for the report!
I discovered one more issue in this area. This OOM condition emulation:
--- a/src/backend/snowball/libstemmer/api.c
+++ b/src/backend/snowball/libstemmer/api.c
@@ -6,3 +6,3 @@ extern struct SN_env * SN_new_env(int alloc_size)
{
- struct SN_env * z = (struct SN_env *) malloc(alloc_size);
+ struct SN_env * z = (rand() % 2 == 0) ? NULL : (struct SN_env *) malloc(alloc_size);
if (z == NULL) return NULL;leads to `make check` crashes like:
2026-08-30 06:42:24.759 EEST postmaster[1423557] LOG: client backend (PID 1423766) was terminated by signal 11:
Segmentation fault
2026-08-30 06:42:24.759 EEST postmaster[1423557] DETAIL: Failed process was running: SELECT
ts_delete(to_tsvector('english', 'Rebel spaceships, striking from a hidden base'), 'spaceship')
or
2026-08-30 06:46:50.112 EEST postmaster[1426347] LOG: client backend (PID 1427391) was terminated by signal 11:
Segmentation fault
2026-08-30 06:46:50.112 EEST postmaster[1426347] DETAIL: Failed process was running: SELECT ts_lexize('thesaurus', 'one');
Could you please have a look if you're still around?
Best regards,
Alexander
Alexander Lakhin <exclusion@gmail.com> writes:
I discovered one more issue in this area. This OOM condition emulation: --- a/src/backend/snowball/libstemmer/api.c +++ b/src/backend/snowball/libstemmer/api.c @@ -6,3 +6,3 @@ extern struct SN_env * SN_new_env(int alloc_size) { - struct SN_env * z = (struct SN_env *) malloc(alloc_size); + struct SN_env * z = (rand() % 2 == 0) ? NULL : (struct SN_env *) malloc(alloc_size); if (z == NULL) return NULL;
leads to `make check` crashes like:
Hmph. SN_new_env itself is visibly okay with this, so the failure is
in some caller. I'm too tired to dig into it myself, but can you
identify the culprit more precisely?
regards, tom lane
30.08.2026 07:04, Tom Lane wrote:
Hmph. SN_new_env itself is visibly okay with this, so the failure is
in some caller. I'm too tired to dig into it myself, but can you
identify the culprit more precisely?
I think it's:
static void
locate_stem_module(DictSnowball *d, const char *lang)
{
...
d->stem = m->stem;
d->z = m->create();
...
(in two places)
then d->z is dereferenced in dsnowball_lexize().
Thank you for your time!
Best regards,
Alexander
вс, 30 авг. 2026 г. в 09:30, Alexander Lakhin <exclusion@gmail.com>:
30.08.2026 07:04, Tom Lane wrote:
Hmph. SN_new_env itself is visibly okay with this, so the failure is
in some caller. I'm too tired to dig into it myself, but can you
identify the culprit more precisely?I think it's:
static void
locate_stem_module(DictSnowball *d, const char *lang)
{
...
d->stem = m->stem;
d->z = m->create();
...
(in two places)then d->z is dereferenced in dsnowball_lexize().
Confirm.
locate_stem_module() in dict_snowball.c stores m->create() into d->z
without checking. *_create_env wrappers correctly return NULL when
SN_new_env fails. dsnowball_init then succeeds with d->z == NULL, and
dsnowball_lexize crashes in SN_set_current / replace_s.
--
Regards,
Rachitskiy Andrey
вс, 30 авг. 2026 г. в 09:37, Andrey Rachitskiy <pl0h0yp1@gmail.com>:
вс, 30 авг. 2026 г. в 09:30, Alexander Lakhin <exclusion@gmail.com>:
30.08.2026 07:04, Tom Lane wrote:
Hmph. SN_new_env itself is visibly okay with this, so the failure is
in some caller. I'm too tired to dig into it myself, but can you
identify the culprit more precisely?I think it's:
static void
locate_stem_module(DictSnowball *d, const char *lang)
{
...
d->stem = m->stem;
d->z = m->create();
...
(in two places)then d->z is dereferenced in dsnowball_lexize().
Confirm.
locate_stem_module() in dict_snowball.c stores m->create() into d->z
without checking. *_create_env wrappers correctly return NULL when
SN_new_env fails. dsnowball_init then succeeds with d->z == NULL, and
dsnowball_lexize crashes in SN_set_current / replace_s.--
Regards,
Rachitskiy Andrey
Fix in attach
Attachments:
t253280_130001-Check-for-NULL-from-Snowball-create_env-in-dsnowball.patchtext/x-patch; charset=US-ASCII; name=0001-Check-for-NULL-from-Snowball-create_env-in-dsnowball.patchDownload+8-1
вс, 30 авг. 2026 г. в 09:43, Andrey Rachitskiy <pl0h0yp1@gmail.com>:
Fix in attach
Looking again at how Snowball is wired in the backend, I was too
quick with the d->z NULL check.
In the backend, snowball_runtime.h remaps malloc to palloc
(src/include/snowball/snowball_runtime.h). api.c includes that
header via the -I order in the snowball Makefile / meson.build, so
SN_new_env()'s malloc is palloc. On allocation failure palloc does
not return NULL. It goes through MemoryContextAllocationFailure().
So the live OOM path never reaches SN_new_env()'s
"if (z == NULL) return NULL" (Or what don't I know?) , never leaves d->z
NULL, and never reaches the SEGV in dsnowball_lexize.
Alexander's change is different. It forces a NULL before the
(remapped) malloc runs:
z = (rand() % 2 == 0) ? NULL : malloc(alloc_size);
That is the Snowball C API's "malloc failed".
The patch I sent only covers that emulated NULL-return case.
It does not change behaviour for a real out-of-memory under palloc.
Sorry for the noise :).
--
Regards,
Rachitskiy Andrey
Andrey Rachitskiy <pl0h0yp1@gmail.com> writes:
In the backend, snowball_runtime.h remaps malloc to palloc
(src/include/snowball/snowball_runtime.h). api.c includes that
header via the -I order in the snowball Makefile / meson.build, so
SN_new_env()'s malloc is palloc. On allocation failure palloc does
not return NULL. It goes through MemoryContextAllocationFailure().
Ah, right. You can confirm that SN_new_env is really using palloc:
$ nm --ext --undef api.o | grep alloc
U palloc
It's like this to prevent memory leaks while not modifying the
machine-generated Snowball .c files, but I concede it's confusing.
Anyway it looks like we have nothing to do here. The Snowball code
is correct on its own terms to defend against null results, but our
calling code is equally correct to not worry about that.
regards, tom lane